Mashup: put your Google Analytics Data inside a Gadget

toggle Cinema 25 Oct 2009

Mashup: put your Google Analytics Data inside a Gadget Posted by Insane in Google, Java, Php | No responses

The acuracy of the data is probably one of the most challenging aspects of the web analysis , on these days to know how to interpretate that data is quite vital for almost every person involved with the web.Google Analytics (’GA’ from now on ) has become one free powerfull tool to track your web campaign.My aim here it is not only to show how easy it`s to remake that GA graphs inside a Google gadget but also to show how usefull they really are outside GA.

For the purpose of this article, i will asume that the reader has previous knowledge on GA otherwise here´s a strongly recommended page for a quick and straight forward initiation. we will use Yahoo Pipes, Google Charts ,Google Gadgets and just a pinch of you server side language.

Note: this is just an experiment, i recomend you to take a look at the Google Analytics Labs if you need a more complete way to get & analize your data.

First of all you need a way to get a perodical report without visit the GA website right ? you need to schedule a report , this can be done by entering to the panel of “schedule report” , complete all fields required by this form and make sure of choose XML as de format of the report ( in order to make this tutorial works, you can change this later ).

Pipe Web Address: http://pipes.yahoo.com/pipes/pipe.info?_id=0ff66029e7bf71cbcd31acbb7279f079

Ok, if anything goes wrong until then you sucessfully had created a scheduled report , the following JAVA code allow you to check your inbox and determinate wheter an email have an attachment and an specific remitent, yes, your scheduled report from GA .

Getting out the report of your inbox.

(toggle java code)

package com.instropy;

import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;
import java.util.Vector;

import javax.mail.*;
import javax.mail.internet.MimeBodyPart;

public class AttachmentDownloader {

 //cyrectory to download the attachments.
 private static final String LOCAL_INBOX = "C:\\ginbox\\";

 public AttachmentDownloader() {
 };

 public static void main(String[] args) {
 }
 /**
 *
 * @throws MessagingException
 */
 public void fetch() throws MessagingException {

 Properties prop = new Properties();
 // Deshabilitamos TLS
 prop.setProperty("mail.pop3.starttls.enable", "false");

 // Hay que usar SSL
 prop.setProperty("mail.pop3.socketFactory.class",
 "javax.net.ssl.SSLSocketFactory");
 prop.setProperty("mail.pop3.socketFactory.fallback", "false");

 // Puerto 995 para conectarse.
 prop.setProperty("mail.pop3.port", "995");
 prop.setProperty("mail.pop3.socketFactory.port", "995");

 Session session = Session.getDefaultInstance(prop);

 String username = "example@gmail.com";
 String host = "pop.gmail.com";
 String password = "******";

 // Get the store
 Store store = session.getStore("pop3");
 store.connect(host, username, password);

 // Get folder
 Folder folder = store.getFolder("INBOX");
 folder.open(Folder.READ_ONLY);

 // Get directory
 Message message[] = folder.getMessages();
 // message.length
 for (int i = 0, n = 5; i < n; i++) {

 Address fromEmail = message[i].getFrom()[0];

 if (!fromEmail.toString().equalsIgnoreCase("fu.wire@gmail.com")) {

 // handle other emails ..

 } else {

 // these are importants

 try {
 Object content = message[i].getContent();

 if (content instanceof Multipart) {

 Vector<EMailAttach> vema = new Vector<EMailAttach>();
 Multipart mp = (Multipart) content;

 for (int j = 0; j < mp.getCount(); j++) {
 Part part = mp.getBodyPart(j);

 String disposition = part.getDisposition();

 System.out.println(message[i].getSubject()
 + " has attachment " + vema.size() + " "
 + disposition);

 if (disposition == null) {
 // Check if plain
 MimeBodyPart mbp = (MimeBodyPart) part;

 System.out.println("EMailAttachName [ "
 + part.getContentType() + ","
 + part.getFileName());

 if (mbp.isMimeType("text/plain")) {

 } else {

 }
 } else if ((disposition
 .equalsIgnoreCase(Part.ATTACHMENT) || disposition
 .equalsIgnoreCase(Part.INLINE))) {

 // Check if plain
 MimeBodyPart mbp = (MimeBodyPart) part;
 if (mbp.isMimeType("text/plain")) {
 System.out.println("Mime type is plain");
 } else {
 System.out.printf("Save file (%s)", part
 .getFileName());
 EMailAttach ema = new EMailAttach();
 try {
 ema.name = urldecode(part
 .getFileName());
 } catch (Exception e) {
 // TODO Auto-generated catch block
 e.printStackTrace();
 }
 File savedir = new File(LOCAL_INBOX);
 savedir.mkdirs();
 File writeContents = File.createTempFile(
 "emailattach", ".atch", savedir);
 ema.path = writeContents.getAbsolutePath();
 ema.size = part.getSize();
 vema.add(ema);
 try {
 ema.size = writeContents(writeContents, part);
 } catch (Exception e) {
 // TODO Auto-generated catch block
 e.printStackTrace();
 }
 }

 }// end disposition if
 }// end for
 }// end if

 } catch (IOException e) {
 // TODO Auto-generated catch block
 e.printStackTrace();
 }
 }// end else
 }
 }
 /**
 *
 * @param name
 * @return
 * @throws Exception
 */
 protected String urldecode(String name) throws Exception {
 if (name == null || name.length() == 0) {
 return "unknown";
 }
 String ret = java.net.URLDecoder.decode(name, "UTF-8");

 // also check for a few other things in the string:
 ret = ret.replaceAll("=\\?utf-8\\?q\\?", "");
 ret = ret.replaceAll("\\?=", "");
 ret = ret.replaceAll("=20", " ");

 return ret;
 }
 /**
 *
 * @param writeContents
 * @param part
 * @return
 * @throws Exception
 */
 protected int writeContents(File writeContents, Part part) throws Exception {

 System.out.println("writeContents " + writeContents);
 BufferedOutputStream bos = new BufferedOutputStream(
 new FileOutputStream(writeContents));

 byte[] buff = new byte[2048];
 InputStream is = part.getInputStream();
 int ret = 0, count = 0;
 while ((ret = is.read(buff)) > 0) {
 bos.write(buff, 0, ret);
 count += ret;
 }
 bos.close();
 is.close();
 return count;
 }

}

Read and interpretate your inform.

Note : now hat we have our xml we need a scheduled cron that copy it to a visible URL , it can be simplest as an a cp command.

At least that you had been hiding down of some kind of big rock on a far far country with no internet connection you must know by now that Yahoo Pipes is the new big thing to compose,manipulate and reorder the information from around the web therefore we give it a chance to manipulate our xml and give it an json output.

There are 6 points to need to be explained about this pipe, these are shown below:

1 -The url of our GA xml.
2 -Extra url parameters if you need it
3 -Fetch Data coming from the URL
4 -Rename of node names.
5 -This step shows how easy it`s to use a regular expression on an specific node , on this case append “Analytics Report”, to the current title.
6- A simple sorter.

Parse it and Graph it.

(toggle php code)

<?php
#*******************************************************************************
function get_url_contents($base_url){

	if (!function_exists('curl_init'))
		trigger_error("curl library is not installed ");
	// initialize a new curl resource
	$ch = curl_init();
	// set the url to fetch
	curl_setopt($ch, CURLOPT_URL, $base_url);
	// don't give me the headers just the content
	curl_setopt($ch, CURLOPT_HEADER, 0);
	// return the value instead of printing the response to browser
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
	// use a user agent to mimic a browser
	curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0');
	$content = curl_exec($ch);
	// remember to always close the session and free all resources
	curl_close($ch);
	return $content;

}

$pid = '0ff66029e7bf71cbcd31acbb7279f079';
$xmlDashBoardAnalytics = urlencode('http://mydomain.com/mygaxml.xml');

$contents = get_url_contents(
	sprintf('http://pipes.yahoo.com/pipes/pipe.run?_id=%s&_render=json&urlinput1=%s',$pid,$xmlDashBoardAnalytics)
);

$ph = json_decode($contents);
$graph = $ph->value->items[0]->Report->Graph;
$chartTitle= $ph->value->items[0]->Report->Title[0]->ProfileName.", ".$ph->value->items[0]->Report->name;
//last 10 days
$points = array_slice($graph[0]->Serie->Point,-10,10);
$chartapi = "http://chart.apis.google.com/chart?cht=p&chd=t:%s&chs=800x300&chl=%s&chco=4d89f9,c6d9fd&chtt=%s";

//google chart
//http://chart.apis.google.com/chart?cht=p3&chd=t:60,40&chs=250x100&chl=Hello|World

$values = implode(",",array_map(
	create_function('$element','return $element->Value*10;'),$points)
);
$labels = implode("|",array_map(
	create_function('$element','$_ = explode(" ",$element->Label); return $_[1]."-".substr($_[3],0,3)." (".$element->Value.") ";'),$points)
);

header("Cache-Control: No-cache, Must-revalidate");
header("Pragma: no-cache");

if(isset($_GET['resultFormat'])):

	switch($_GET['resultFormat']):

		#***************************************
		#SWF
		#***************************************
		//can be used XML/SWF Charts by example
		#***************************************
		#DEFAULT , IMAGE TAG
		#***************************************
		case "image":
		default:
			header("Content-Type: text/plain");
			$url_image = sprintf($chartapi,$values,$labels,$chartTitle);
			die("<div style='border:1px solid #CCC; padding:5px;>'<img src='".$url_image."'/></div>");
		break;
	endswitch;

endif;
#*******************************************************************************
?>

All the requirements to make things works are in the basket, now it`s just a question of time to put them togheter, next steep Google Gadget ( God bless the GG )

, this is the simplest of all steps.The main reason why i decided to choose Google gadgets is that it`s more and more easy to place them in

All the requirements to make things works are completed, now it`s just a question of time to put them together, next steep is the gadget ( God bless Google Gadget), this is the easiest part of this tutorial.

The main reason why I decided to choose Google gadgets is because is more and more easy to place them in almost all the services of Google.

The gadget

(toggle Google Gadget Code)

<?xml version="1.0" encoding="UTF-8" ?>
<Module>
<ModulePrefs title="Mashup Analytics Google Gadget"  height="300"/>
<Content type="html">

<![CDATA[

<div id="content_div">
	<span>loading...</span>
</div>

<script type="text/javascript">	

	function makeGETRequest(url) {
		var params = {};
		params[gadgets.io.RequestParameters.METHOD] = gadgets.io.MethodType.GET;
		gadgets.io.makeRequest(url, response, params);
	};

	function makeCachedRequest(url, callback, params, refreshInterval) {
	  var ts = new Date().getTime();
	  var sep = "?";
	  if (refreshInterval && refreshInterval > 0) {
		ts = Math.floor(ts / (refreshInterval * 1000));
	  }
	  if (url.indexOf("?") > -1) {
		sep = "&";
	  }
	  url = [ url, sep, "nocache=", ts ].join("");
	  gadgets.io.makeRequest(url, response, params);
	}

	function response(obj) {
		document.getElementById('content_div').innerHTML = (obj.text);
	};

//serverside call
makeCachedRequest("http://www.yourdomain.com/serverside.php?resultFormat=image",response,{},60);

 </script>
]]>
</Content>
</Module>

Final Result

Even if this could be considered as an looong workaround for one solution , you must admit is funny to create our own graphics and refine the results of it isn´t it ? I hope someone find this usefull . In the image you can see how easy it`s to integrate this gadget to Google Wave and Gmail by example