Duct Tape Programmer

Google+ Changed, G+7 Had To Be Fixed

2011-12-07T05:08:00.001-08:00

I have noticed a couple of days ago, that very important functionality in the G+7 desktop gadget stopped working. You could no longer view and add comments, add +1s or share.

Some investigation revealed that Google changed they way they handle dynamic content of Google+. It was a pretty nasty change, as the initial data is no longer included in the page content as a static table, but dynamically created in parts, using Javascript. This made my post about parsing Google+ post pages obsolete. As well as broke G+7.

Fortunately, I managed to reverse-engineer what Google does yet again and prepared version 1.8 of G+7 that brings back the missing functionality.

Go here to download it and enjoy.

I am going to prepare a blog post soon, outlining what Google is doing now on their Google+ pages and how you can parse the new format of the data.

Are We Done Yet? Or - Earned Value Calculations in Agile Projects

2011-11-03T14:12:00.000-07:00

Don't worry, this is not one of these posts where someone attempts to cram Scrum burn-down charts into waterfall CPI/SPI to satisfy upper management who don't speak agile. This post is going to describe a metric that can be useful to project managers running software projects using agile methodologies. The metric is going to tell you how close to "done" you are in your project. Or more precisely: how close the value delivered by your project so far is to the total expected value of the project.

I have been using agile software development methodologies for a few years now. One thing always surprised me - while agile has effort tracking figured, described and codified as a set of useful practices, tracking earned value for projects is lacking. Up until recently, it was hard to find any mention of agile value tracking on the internet. These days, some authors write about it, but somehow the subject is not very popular in agile teams.

The Confusion

I have witnessed time and again that project teams tend to confuse effort spent (or estimated) with value delivered (or promised). What the popular metrics (like agile burn-down charts or traditional CPI/SPI values) do measure is effort, time and budget spent, and how they compare to the plan. And while very useful, effort metrics are not equivalent to value metrics. Not even close to equivalent to be exact. Why? Quite simple. Sometimes features that are very valuable are not very costly. Sometimes the opposite happens - you are working on something that requires a lot of work, but the return on investment is questionable. In a typical sofware project, a relation of a cost of a feature to its value is vary, very non-linear. Yet, the fallacy of equaling effort spent (or, in other words, sunken cost) to value delivered is very very common.

Why is knowing the value delivered useful? Well, it lets you decide whether it is worthwhile to continue the project or not. In fact, under normal circumstances, this metric should be the metric deciding the fate of your project. It is much more important than budget, time and effort overruns: you will want to continue a project that continues to bring you satisfactory amount of value regardless of whether it costs more than you expected or not. And you will want to cancel a worthless project even if it is cheap, because you can redict your people and resources to more promising undertakings.

Value Points

To measure value delivered by your project as you keep working on it is to add one more parameter to each of your user stories. The metric is called Value Point (VP). Value Point is vary much like a Story Point (used for effort estimation and tracking), in that it is relative. It tells you how the value of a story compares to the value of some other story. Which is useful, because it lets you estimate stories for value using not just concrete financial metrics, but also more fuzzy things (like gut feeling).

Who assigns the Value Points? The ones responsible for this are project sponsors and any other party that have vested interest in the outcome of the project, can benefit from it, or can provide input for a value of particular story. How is the value points assignment done? the simplest way is to use an equivalent of Scrum release and iteration planning poker and calculate the average of VPs assigned by each participant of the planning meeting. A Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, 34, 55, 89...) is used to establish "legal" values. That is the methodology that is suggested by majority of texts on the subject.

However, other methods can also be used and are sometimes more adequate. For example, there is nothing sacred about using Fibonacci sequence here. You can use a continuum of values, fractional values, span many orders of magnitudes (this is a no-no for Story Point-based effort planning, but for value estimations, it is not uncommon to be able to tell with confidence that something is 200x more valuable then something else in the same project). Best of all, as long as the decision-making team is stable, you don't really need to reach any sort of a consensus and to pick an average Value Points number. Instead, you can just calculate a total out of everybody's assigned Value Points and use this as a final VP number for a story (in contrast to this, in effort planning, a consensus is absolutely required)

What Are Value Points Good For?

After you assigned value points to stories in your project, you are able to provide very useful metrics to the project management team

is the project still delivering as much value as it did when we first started it?
If the value stream dropped, is the value being delivered high enough to justify continuing the project?
what is the percentage of "very valuable but easy to implement" features? To establish that, you pick stories that have high VP metrics and low Story Point metrics
what is the percentage of "not very valuable but hard to implement" features? If you have stories like that, you may want to consider dropping them (or maybe postponing until they become more valuable)
last but not least: how "done" are you in the project - meaning - what percentage of value points has been completed in the project. Upper management very often craves for this sort of a metric

Want to record your desktop? Try DeskBeam Junior

2011-10-24T01:46:00.000-07:00

Sometimes you just want to scratch that itch. For some time, I have been wondering what would it take to write a screen recording application. For Windows. The existing ones were either too primitive, or too expensive.

All the freeware recorders that I could find recorded either with lossy compression, which made the resulting video stream look awful (not to mention they dropped frames like crazy at higher video rates, because MP4 compression is CPU intensive), or with no compression at all, which made the recorded files huge (like 1GB per minute huge).

Then there were the "it's free, but" variety: free, but limited redording length (varying from 15 seconds to 5 minutes), free but with watermark added to the video, or free but only work with the vendor's own video sharing web service. None of these was good enough for my liking.

And last but not least, there are the expensive recorders with all sorts of bells and whistles. Good products, but costly.

I wanted to have something modest and simple (but not primitive), which would be able to capture desktop activity with 100% fidelity, apply a decent compression, so that the recording has sane size and which woudl allow me to perform some basic editing, such as

trimming the beginning and the end of the recording
fading in and out
title and credit screens
watermarking
adding background audio

Plus I wanted to be able to compress the result to WMV (remember, we are talking Windows-based recorder here) and upload it to YouTube for the world to see.

The work took me a good while, because the problem turned out to be more difficult than I thought. After I solved the main obstacles of recording the video, all kinds of details to address popped up (there are always problematic details to address, aren't there?).

But eventually the project was finished. You can download the DeskBeam Junior installer from http://www.kalamonsoft.com/dbjr and try it out.

Or take a look at some screenshots to whet your appetite

Recorder dialog

Player/editor

Compressor

YouTube uploader

Using Google+ API - The Real One - Part 2. Ajax, Paging and Commenting

2011-10-21T03:41:00.000-07:00

In the first installment of my blog series about using the real Google+ API, I have described how to retrieve a Javascript structure, representing messages by parsing a Google+ page source. This was a good introduction, buty there is a better way to retrieve a message stream - by calling Google+ server directly using Ajax calls. This method of communicating with Google+ is better than screen scraping described before, because it allows you to get more than the first page of messages

The Message Stream Ajax Call URL

In order to retrieve a message stream from Google+, perform a HTTP GET request to this URL:

https://plus.google.com/_/stream/getactivities/?sp=[1,2,null,circleid,null,20,null,"social.google.com",[]]&rt=j&ct=token

In order to overcome the potential problem with caching (Internet Explorer tends to return cached version of the server response instead of actually contacting the server), you will want to append a dummy timestamp parameter to the request, like this:

'&_requid=' + new Date().getTime()

There are three important parameters of the call, which let you control what gets returned. I have marked these in red color in the URL above.

The first one - circleid - is the identifier of the circle, from which you want to retrieve the message stream. If you want to retrieve messages from "all circless", set this to null. Otherwise, you will have to know the ID of the circle you want to retrieve messages from. I will show you how to retrieve information about circles in the next installment of this series
the second parameter in red is the page size. The default is 20 messages, but you can set this to something else, depending on your needs
the third parameter - &ct=token - is the "page token" of the page of message stream that you want to retrieve. This is an optional parameter - you should omit it if you want to retrieve the first page. Otherwise, you have to know what the page token of the page that you want to retrieve is. I will describe this in more detail below.

Retrieving the Message Stream

Given the URL above, you can execute the following jQuery to retrieve message stream:

var frontGarbage = "Invalid JSON: )]}'";

jQuery.support.cors = true;

jQuery.ajax({

type: 'GET',
url: url,
error: function (xhr, status, error) {
if (error.indexOf(frontGarbage) == 0) {
var response = jQuery.parseJSON(error.substring(frontGarbage.length));
}
}
});

Before we dig into what is the content of response , I first need to explain two things:
why is the response retrieved in the error handler and not in the success handler of the jQuery Ajax call?
what is frontGarbage?

The answer is - the first is the consequence of the second. Google+ server, in all its wisdom, prefixes the proper JSON response of the message stream (and pretty much of any response that it returns to the callers) with the garbage text. This is probably an attempt to make my life more difficult. As a result of this, jQuery is unable to automagically parse the response as JSON and decides that the server errored. Fortunately, it gives us the full server response in the error parameter, allowing us to parse the JSON manually

Now, on to the response data. The majority of it, namely response[0][1][1][0] is the message table containing messages in the exact same format as described in my previous post. You can interpret it in the same way, no modifications required.

There is an additional important parameter in the response: response[0][1][1][1]. It is a pretty long string of hexadecimal characters, which constitutes the "next page token" value for a page that you have just retrieved. Which means that if you pass this value as the ct parameter of the Ajax call's URL, you will get the next page of messages. Curiously, there does not seem to be a "previous page token" anywhere in the response. Which means that you have to remember the already retrieved tokens in some kind of a table, and use the correct one to move back and forth between the message stream pages

Adding Comment To The Post

Now let's concentrate on a stuff that is actually absolutely impossible in the current incarnation of the Google+ API. Which is - let's modify something in the message stream, which the official API does not let you do, because it is a read-only API.

Before we begin - you may be wondering now, why I have described two methods of retrieving a message stream: by screen scraping the HTML page, and by means of an Ajax call. The reason for this is that in order to perform any sort of non-read-only operation, you need to know the XSRF token. Otherwise, Google+ server will reject the call (and rightfully so - we don't want Google+ to be insecure, do we?). For obvious reasons, the XSRF token cannot be obtained by an Ajax call - it has to be contained on a page. So to get to the token required to modify a message on Google+, you have to do the following:

obtain the URL of the message (the post[21] described inthe previous post contains the URL - you will need to prepend https://plus.google.com to this value)
screen-scrape the page to obtain its message data, as described in the previous post
the pageData[1][15] is the XSRF token that you want

After every non-read-only operation, you will hav eto obtain a new XSRF token again. The simplest way to do it is to repeat step 2 and 3. But typically, the next XSRF token is also present in all server responses from state-modifying operations. Identifying these is left as an excercise to the reader (Firebug is your friend here).

Now that we have the required XSRF token, posting a comment to the Google+ message is very easy

jQuery.support.cors = true;

var data = 'itemId=' + postId + '&text=' + encodeURIComponent(text);

data = data + '&clientId=os:' + postId + ':' + new Date().getTime();

data = data + '&at=' + encodeURIComponent(xsrf) + '&'

jQuery.ajax({
type: 'POST',
url: 'https://plus.google.com/_/stream/comment/?_requid=' + new Date().getTime() + '&rt=j',
data: data,
error: function (xhr, status, error) {
if (error.indexOf(frontGarbage) == 0) {
// this is a success
} else {
// this is a failure. You may want to display some error message
}
}
});

The important elements of this code are

postId - this is the post ID described in the previous post. It is contained in the post[8] element of the message structure
text - this is the text of the comment that you want to post

To Be Continued

In the next installment, I will describe how to:

retrieve a list of circles
add or remove a +1 to the post

Using Google+ API - The Real One, Not The Official Ersatz. - Part 1. The Basics

2011-10-17T01:47:00.000-07:00

Important Update

hile This post is now a little bit obsolete, because Google changed the way they render Google+ post pages. I am going to prepare an updated post describing the changes very soon. Meanwhile you can go to my announcement post to grab the latest version of G+7 that fixes the problems that Google's changes caused.

Introduction

While working on the G+7 Windows Desktop Gadget, the most difficult problem was retrieving information that I wanted from Google+, and also, updating Google+ with stuff (like adding comments, sharing, +1-ing, etc.). When I started, there was no Google+ API available at all. People all over the world were trying to find some workarounds, but none of them was good enough to be of practical use.

Since then, Google released a first version of Google+ API. Unfortunately, in its current incarnation, the API has a very limited functionality. As Steve Yegge described it in his recent infamous rant, it is a "Stalker API" - it allows you to learn stuff about a particular person on Google+ - their posts, circles, comments, etc. But it does not let you do any of the stuff you would want to use to implement a proper Google+ application, such as a standalone desktop client, a mobile client, a desktop gadget etc. There is no way to retrieve a stream of activities that you as a Google+ user see when you open http://plus.google.com. The is no way to comment, share posts, +1 a post or a comment.

I hope this situation changes (Google, are you listening?), but in the meantime, I had to resort to reverse engineering the protocol that Google+ pages actually use to produce the content that you can see on them. The result of this reverse-engineering efforts will be the subject of the series of blog posts that I am beginning today. I hope to post every second day. I hope that these posts are useful to Google+ developers out there.

Todays post will cover some very important basics, such as data fromats, protocols, etc. Next installments will describe the following topics

details of authentication
retrieving list of your circles
retrieving message stream for a particular circle
adding +1 to a post
commenting on a post
sharing a post

The list above reflects the current functionality of G+7 desktop gadget. Which means that it reflects the state of reverse engineering which I am currently at. There is obviously more to Google+ than that, and I suppose over time G+7 will be updated, as soon as I figure out details of of other operations.

The Basics - Authentication, Protocols, Data Formats

Google+ uses Ajax to create pages that you see. What happens when you open https://plus.google.com (or any other page containing at least one message on Google+) is - in order

a page with some trivial HTML is loaded and displayed
the page, in addition to HTML, contains a Javascript data structure that encodes the details of the currently logged in user, as well as a first page of the message stream that you see
this structure is read by Javascript code and HTML that displays message stream is dynamically constructed by Javascript
Javascript executes some HTTP GET and HTTP POST calls to Google+ to retrieve additional information and put it on a page

The easiest (but not the only) way to start implemeting a Google+ client is get at this first page and retrieve the initial Javascript data structure. For this, you have to

issue a HTTP GET request to https://plus.google.com or to a page with one particular message
read in the resulting page
find the part of the page between the string 'OZ_initData = ' and ';window.jstiming.load.tick('idp');'
interpret this substring as a Javascript data structure
use the structure for subsequent operations

This is how this can be done using jQuery, like this:

jQuery.support.cors = true; // this is a secret sauce, which I will describe later
jQuery.ajax({
        type: 'GET',
        url: 'https://plus.google.com/?_requid=' + new Date().getTime(),
        success: function (data, status, xhr) {
                var start = data.indexOf('OZ_initData = ');
                var end = data.indexOf(";window.jstiming.load.tick('idp');");
                if (start > 0 && end > 0) {
                    start += 'OZ_initData = '.length;
                    var pageData = jQuery.parseJSON(data.substring(start, end));
    // ... do stuff with pageData
                }
        },
    });

You may be wondering what's the deal with jQuery.support.cors = true;. Well, to be frank - it is a hack. The propblem with Ajax support in jQuery is that it only works within a single domain. Which means that only Javascript loaded from a particular web page is able to access URLs belonging to a domain, to which the page belongs. Which means that if you implement a desktop gadget or a desktop client, you are out of luck - jQuery will throw a "No Transport" error at you if you tell it to make an Ajax call to https://plus.google.com. Unless of course, you force it to behave. The trick was described in this excellent post. But admittedly, this is a trick and a hack, and if jQuery implementation ever changes, it might not be possible to do this any more

Side note - Authentication

As you can see, the data structure that you want to get at is contained now in a pageData variable. I will describe this data structure in more detail later, but first we must overcome one big hurdle. Namely - in order to be able to view https://plus.google.com, you need to authenticate. If you are implementing some code that will execute in a browser, in the context of an already logged in user (like a greasemonkey script, browser plugin, or somesuch), there is nothing to be done - you are already authenticated and jQuery will execute its Ajax request with proper credentials. But if you are implementing something like a standalone client - say a .NET desktop client for Google+, you must first retrieve three cookie values, resulting from authenticating to Google+. The names of cookies you want are: SID, SSID and HSID. Then you have to use these cookie values in the Cookie header of all your HTTP requests. I will cover how to get these cookies in an example .NET application in one of the subsequent posts.

On To The Good Stuff

The data structure that we retrieved to the pageData variable has been partially described in an excellent post that I used as a reference when implementing G+7. That post took a stab at figuring out the data format. It was a tremendous help, but it missed some less frequently used details. I have discovered some more information in there. Here it goes

for a page with a list of posts (like https://plus.google.com or a page showing a stream from a circle)

post = pageData[4][0][i] - where i is the post index. There are 20 post in the table

for a page with a single post (like this one)

post = pageData[20]

Each post is a huge table (93 entries, each of the cells can have big sub-entries). Details of each post:

post[3] = poster name
post[4] = Full HTML text
post[5] = post timestamp (milliseconds since 01.01.1970)
post[7] = Comments post
post[7][x][16] = comment author photo. You will want to add a '?sz=16' parameter to shrink the image
post[7][x][6] = comment author ID
post[7][x][1] = comment author name
post[7][x][3] = comment timestamp (milliseconds since 01.01.1970)
post[7][x][2] = comment text
post[8] = ID of the post
post[11] = Array of one or more links
post[11][x][3] = Title of link
post[11][x][5][1] = URL of image uploaded
post[11][x][21] = Description of link
post[11][x][24][1] = Linked URL
post[11][x][24][4] = Type: document, image, photo, video
post[11][x][41][0][1] = Thumbnail of image
post[16] = poster ID
post[18] = URL of the poster photo. You will want to add a '?sz=48' parameter to shrink the image
post[21] = Link to Google+ page for the post
post[25] = shares of the post
post[27] = location where the post was made
post[27][10] = a miniature image of Google Maps image of the location, centered at the location coordinates
post[27][3] = name of the location
post[27][8] = Google Maps URL of the location
post[44] = if the post is a share of some other message, the original message author
post[44][0] = original message author name
post[44][1] = original message author ID
post[44][4] = original message author photo URL

post[47] = if the post is a share of some other message, the message accompanying the share
post[73][9] = if you are one of the people who +1d the post, this is set to your name
post[73][16] = number of +1s for the post
post[93] = number of comments

Some potentially useful details for dealing with the information above

URLs don't necessarily start with http: or https:. This is ok if you use them from the context of the page you are currently on (e.g., if you are implementing a greasemonkey script). But if you implement a standalone client, you may want to fix these URLs by prepending https: to them before use
The user mugshot photos come from Picasaweb. As such, they are quite big, but if you append a '?sz=48' parameter to them , they get resized to 48x48 pixels. This also works for 16x16. I have not tried other sizes, but I suppose they work also
Other image URLs can come from anywhere. And suppose you want to display them shrinked proportionally, you will need to have a way to do it easily. I found that Google+ uses some internal service to do so, which I also taken the liberty to use. The following Javascript function does the trick:

function getSizedImage(imgUrl, size) {
return
'https://images1-focus-opensocial.googleusercontent.com/gadgets/proxy?url='

+ encodeURIComponent(imgUrl)

+ '&container=focus&gadget=a&rewriteMime=image/*&refresh=31536000&resize_w='

+ size + '&no_expand=1';
}

To Be Continued

Well, I guess this is enough for today - like I said, the message structure is quite huge and describing it took a lot of space.

In the next installment, I will describe how to:

interpret some additional information that can be retrieved using Ajax requests
retrieve the same information as above using Ajax request and not screen-scraping the Google+ web page
display more then one page of messages (20 posts)
comment on a message

Sharing posts from G+7 desktop gadget coming your way

2011-10-13T12:33:00.000-07:00

As I promised several times, after a couple of nights of heavy reverse engineering, I have implemented sharing posts in G+7 desktop gadget. Currently you can share eithe rpublicly, or with one selected circle. I suppose I could add more sharing possibilities if there is enough interest (+1 this post and/or comment if you are interested)

You can also type in a custom share message

~~The functionality will be available tomorrow, when I release version 1.7.0.0. Stay tuned.~~

UPDATE: Version 1.7 with support for sharing is avalable. Go get it!

In the meantime, take a look at some screenshots

More on +1 button in G+7

2011-10-11T10:38:00.000-07:00

So Google must have changed something again, because as of today, the +1 button in G+7 works properly again - i.e. it updates its state properly based on refreshed information from Google+. Go figure.

I will be fixing this tomorrow-ish anyway, because I don't trust Google any more :)

Plus I will try to work some more on sharing posts

Stay tuned

G+7 - +1 button does immediately update its state after adding +1 to a post

2011-09-28T09:29:00.000-07:00

I have just noticed that something changed in the Google+ infrastructure and the +1 button in the G+7 desktop gadget does not properly update its state after adding +1 to a post, because there seems to be a significant delay in propagating the added +1 to the message stream. The +1 does get added, but the button stays white instead of turning blue. Refreshing the message a couple of times fixes things.

This used to work when I implemented this functionality (the update was immediate), so something must have changed at Google.

Anyway - fix is coming with version 1.7, which is going to be available very soon (probably after weekend). I am going to:

add a delay before updating the page
flip the +1 button state manually after update without waiting for the message stream update

G+7 version 1.6 out - authentication problems fixed

2011-09-23T00:57:00.000-07:00

Since I removed all .NET components from G+7, I started receiving bug reports from people who could no longer log in to Google+ with my gadget. What was really weird - I was unable to reproduce this on any of my Windows 7 machines, and I have quite a few of these, in all sorts of flavours - 32 bit, 64 bit, on laptops, desktops, workstations, whathaveyou.

Anyway, the problem started to look serious, as quite a lot of people seemed to have this problem.

My assumption was that the gadgets share cookies with the Internet Explorer, which the documentation at MSDN seemed to imply (it is not a very good documentation, mid you, there is a lot of guessing involved). So if you logged in to Google+ with Internet Explorer, the gadget gets logged in too.

Apparently, this is not entirely true, and there are circumstances where the cookies are not shared.
They do seem to always be shared on 32 bit windows. On 64 bit, not quite. I happened to have "Internet Explorer 9 Preview" installed on my 64 bit machine, and it worked. But when I uninstalled it (on a hunch), suddenly I too got locked out of Google+ in my gadget.

So the question was - what's going on? And most of all - how do I let people log in reliably? After much head-scratching it started to look like the gadget is not exactly tied to Internet Explorer. What it does get tied to is your .NET platform. So if you run a .NET application that has "Web Browser" control embedded in it, you point this at plus.google.com and authenticate with it, then the gadget is let in too.

Well, how weird. But, in absence of useful Google+ API (the current attempt is very very lacking - Google, are you listening?), this is the only way.

So, I have released version 1.6 of the G+7 gadget, which brings back the .NET based authenticator. It is more lightweight and does not attempt to log you in automatically, but it seems quite reliable.

Try it here: http://www.kalamonsoft.com/serve?file=gp7/1.6.0.0/gplus7.gadget

Twitter "Tweet" button does not count tweets? You may be doing it wrong :)

2011-09-22T01:45:00.000-07:00

Some time ago, I decided to add the Twitter "Tweet" button to subpages of kalamonsoft.com, so that you could share your feelings about the software I make. The problem was - the button was not counting tweets. Now, it could mean that nobody cares to share their opinion about my products on Twitter, but it was unlikely, as Facebook and Google+ counters did go up.

Searching google did not help at all. I was just about to give up on this button, but I found this Twitter development discussions place, where you could ask questions and hopefully somebody listens. So I asked my question, and lo and behold, @episod answered (praise this guy).

The problem was - in order for the button to work, the HTTP HEAD request has to be properly handled by your site. Now, for most of the self-hosted sites this is not a problem as web servers have this built-in. But I am hosting my site on Google App Engine, and the site is implemented in Python. I am providing handlers to HTTP requests by overriding some methods of the webapp.RequestHandler class. Typically only the get() or post() method is overriden. Which makes HTTP HEAD not work - an error is returned and the twitter button does not like this.

After I added a trivial head() method override, the twitter button immediately started to work.

So, if you want the twitter button to work on your site, remember to handle HEAD requests properly.

DeskBeam Junior screencasting app - new version available

2011-09-20T10:32:00.000-07:00

Today I have released new early access preview version (0.9.3) of DeskBeam Junior screencasting application. This version is pretty close to what the actual final 1.0 release will look like. There are a couple of minor bugs with background audio trimming left, but they don't affect final (WMV-compressed) capture files.

The big features of this EAP are

screencasting and screen captures from secondary monitors
fixes for captures at increased DPI sizes (important for folks with bad eyesight :))
autoupdate fixes
a bunch of other bugfixes

As you can see, visually not much changed, other than the addition of the monitor selection dropdown. But there is quite a lot going on under the hood.

I have also added a "welcome" message in the player. Not a big deal, just a cosmetic change.

As usual, you can download new DeskBeam Junior version from here

Trying to implement sharing posts - more fun reverse-engineering Google+

2011-09-19T03:03:00.000-07:00

As promised in my previous post, I have just released G+7 version 1.5, which supports adding +1s to posts.

With this out of the way, I am starting to investigate how to implement sharing posts. Of course, this is not (yet) possible with the freshly released Google+ API, so as usual, I have to do some reverse-engineering and add a lot of guessing. I am close to understanding how everything works, with selecting circles to share with and all the gory details. There is just one tiny detail that I have no idea about. So, Googlers, hackers, Ajax people, maybe you van help me out.

Apparently, to share a post on Google+, I have to POST this sort of JSON message to it:

[
    "share with nobody",
    "oz:104167973823715920379.13280d46e2b.0",
    "z134ip5j0ovce3a4a04cgvszxu2qdjrgk24",
    null,
    null,
    null,
    [],
    null,
    {
        "aclEntries":
            [
                {
                    "scope":
                        {
                            "scopeType":"focusGroup",
                            "name":"empty",
                            "id":"104167973823715920379.4c320e48a4b5a4a",
                            "me":false,
                            "requiresKey":false,
                            "groupType":"p"
                        },
                    "role":20
                },
                {
                    "scope":
                        {
                            "scopeType":"focusGroup",
                            "name":"empty",
                            "id":"104167973823715920379.4c320e48a4b5a4a",
                            "me":false,
                            "requiresKey":false,
                            "groupType":"p"
                        },
                    "role":60
                }
            ]
    },
    true,
    [],
    false,
    false,
    null,
    [],
    false,
    false
]

Believe it or not, but I know what all the above long numbers are (104167973823715920379 is some sort of category marker or whatever that can be retrieved from another Ajax call, 4c320e48a4b5a4a is the circle ID and z134ip5j0ovce3a4a04cgvszxu2qdjrgk24 is the post ID). The only thing I have no idea about is the stuff in the red. It looks like some sort of a timestamp or something, but unlike regular Google+ timestamps (which are just new Date().getTime() Javascript fuction results), this one has some hex numbers in it (or so it seems).

So, if you are able to help me out here, just let me know.

Or, if you are a Googler working on the Google+ API, feel free to just implement this functionality in the next rev or G+ API. I would really be grateful if I could just dump this stupid reverse-engineering stuff and use a proper API.

UPDATE (12 Oct 2011):

the mysterious part kinda looks like timestamps (milliseconds since 00:00 01.01.1970) in hex.

13280D46E2B is 1316421398059 in decimal, which is some time on Sept 19th 2011, which is when I conducted the experiments and when the post was made.

Will I be able to implement sharing soon? Find out tomorrow

+1 button in G+7 strikes back

2011-09-16T04:35:00.000-07:00

Well, a second post today, just to let you know that the +1 button in G+7 desktop gadget is coming along nicely and will be available on Monday, when I release version 1.5. Stay tuned

UPDATE: G+7 version 1.5 (with the +1 button and other good stuff) is available here: http://www.kalamonsoft.com/gp7

Another Day, Another Release - G+7 version 1.4 is out

2011-09-16T00:45:00.000-07:00

G+7 has been using an external, auxiliary application (googleauthenticator.exe) to perform authentication in Google+. This caused quite a few problems. The external application introduced dependancy on .NET framework. Some users with non-latin languages reported problems with authenticating, which was caused by the authenticator using heavy screen-scraping techinques to perform authentication. Some virus scanners (esp. Norton) flagged the authenticator as a potential threat, as it launched from a non-standard location (gadget installation directory).

But worst of all, the authenticator was a total hack and a worakround for not being able to properly log in to Google+.

Well - the authenticator is no more. Starting with G+7 version 1.4 (released a minute ago), G+7 authenticates using your regular Internet Explorer credentials. It became more lightweight, it does not depend on any external frameworks, it behaves better. Enjoy.

As usual, you can get G+7 from here

G+7 version 1.3 - with support for adding comments. And problems with Microsoft Live Gallery

2011-09-15T08:03:00.000-07:00

For the last week, I have been working hard on providing support for adding comments to posts from within the post view in G+7.

This required quite a lot of effort, as I had to do a lot of reverse-engineering of Google+ Ajax calls. First - to dynamically reload the post after the comment is added and second - to actually post a comment. It took me a while to figure out everything and put all the pieces together, but eventually I was successful.

So this is what the result looks like

I hope you will like the new functionality as much as I do.You can install G+7 by following this link: http://www.kalamonsoft.com/gp7By the way - for a short while, G+7 was available on Microsoft's Windows Live Gallery. But I have just noticed that something broke, and G+7 download link no longer works. I am not sure what exactly broke the link, but I took the gadget offline, until the problem gets fixed. I submitted an update and hopefully things will get corrected by Microsoft people. However, it seems to be taking them a while to respond to update requests (last time it was around 2 weeks). Please stay tuned.

DeskBeam Jr error - please contact me

2011-09-10T05:56:00.000-07:00

To whomever anonymously posted a bug in our bugtracker about the crash in DeskBeam Junior - please try to install Microsoft runtime redistributable package from http://www.microsoft.com/download/en/details.aspx?id=5555 and try again

Ah, and contact me :)

Thanks

UPDATE: the error has already been fixed

Over Quota at Google Apps Engine :)

2011-09-08T11:19:00.000-07:00

Oh dear. You guys are amazing me. Suddenly so many people started to download G+7 that my resource usage at Google Apps Engine went over free quota. Who knew it would happen on the third day :)

So, the site is intermittently unavailable. I have enabled billing and we should be back online in 20 minutes. At least this is what Google says.

Sorry about the mess

G+7 version 1.2 - now with support for Circles

2011-09-08T07:24:00.000-07:00

In my last post, I mentioned that I have been working on providing support for Google+ circles. Well, today I have released G+7 version 1.2, in which circles support became available for public consumption.

Here is how circles work in G+7
After logging in, if you want to limit the stream of updates to a certain circle, simply select this circle from a dropdown in the settings panel, as shown in the illustration to the left.
If you select a circle, and additional icon (a sort of b/w mini "Circles" icon from Google+) shows up in the gadget's header. When you hover your mouse over the icon, it shows you the circle name in the tooltip. When you click on the icon, it opens the circle's feed in your browser.

You can get version 1.2 of G+7 here. Try it and let me know what you think

Plans
I have big plans for the future. I will be adding support for commenting, sharing posts and for adding +1s to them. Stay tuned. You can find information about updates in the blinking icon in the gadget itself, or on twitter

G+7 And Google+ Circles

2011-09-07T07:48:00.000-07:00

I am working on G+7 version 1.2. The big feature of this version is going to be the ability to filter the message stream by circles.
The feature is coming along nicely.
Actually it seems to be much easier than I expected, because it looks like jQuery is very much able to properly talk to Google+ with valid credentials, if I prod it sufficiently hard, so I can get rid of most of the .NET-based helper apps

Anyway - stay tuned

UPDATE: support for circles is now available with G+7 version 1.2

G+7 Google+ gadget - version 1.1.0.0: fun with +1 button

2011-09-06T04:53:00.000-07:00

Today I have released version 1.1.0 of the G+7 Windows desktop gadget.

The original plan for this release was to add the +1 button to every message, so that you could quickly +1 a post without having to open it in the browser. Unfortunately, the reality proved to be more complicated than my hopes. It seems like the button, while working fine for web pages (like this one), does not really behave properly for posts on Google+ itself. It does render the number of +1s correctly, however clicking on it, and passing the post's URL as the href parameter (which is an URL like this one) does not increase +1 count on the post page. I am not exactly sure what clicking on the button actually does, but it is not what one would expect.

So, instead, I have provided just a read-only number of +1s on a post, and a note to remind you if you were one of the +1ers or not (sorry CmdrTaco for using your post for illustration purposes. I hope you don't mind)

The +1 button functionality will come later. To implement it, I have to reverse engineer the Ajax call that Google+ itself uses to bump the +1 on a post (I have a pretty good idea about how to do it) and create some mini-app in .NET invoked from teh gadget to do the bumping.

Either that or some jQuery trick may be possible as I have noticed that once I perform the login from a helper app that contains an embedded IE control, the gadget itself seems to become aware of all auth cookies that Google+ requires.

Anyway - the lack of API for Google+ is a royal pain in the arse and a real shame. I hope Google gets its act together soon, as having to reverse-engineer inner workings of G+ web pages to implement any standalone G+ client is not a good long-term strategy

G+7 - Google+ Windows Desktop Gadget

2011-09-05T02:33:00.000-07:00

Last week, I created and released G+7 - a Windows 7/Vista desktop gadget that displays live stream of updates from your Google+ friends.

The implementation was a bit tricky, first of all because of the face that Google+ does not offer any sort of official API. So I had to resort to reverse engineering Ajax calls made by Google+ web page to fetch posts. Second complication was caused by the face that Windows desktop gadgets are pretty much pure javascript, and because of that they have trouble creating and using authentication cookies that Google uses - like I said, there is no API available to authenticate you programmatically to Google+, so some hacks had to be made, some .NET code used to retrieve cookies as well as fetching update streams.

All of this makes G+7 a bit of a hack job.

But anyway - I suppose it is quite useful to some people, as I have been seeing quite a lot of downloads over the weekend.

I am planning some additional fucntionality in the future:

posting comments to messages
sharing posts
+1 button

Give it a try

Here are some screenshots

"None of this affects me"

2010-12-07T03:25:00.000-08:00

Great comment on slashdot:

‎"I'd really like to comment on this but I afraid of the consequences. I'd like to work someday and possibly travel to the US. I'd rather just pretend I don't know what's happening. Besides, none of this really affects me. It's about the past and from where I stand today nothing from any of the actions they have taken has changed my life in any way. At least now yet."

My email to a dude I know at Twitter about WikiLeaks

2010-12-06T06:05:00.000-08:00

I care a lot about WikiLeaks. It is hard not to. I also care a lot about the fact that despite obvious volume of tweets on Twitter, with #wikileaks hashtag, they do not show up as "trending" Twitter's, making it seem like the subject is not relevant to a significant portion of the audience - which, as it happens, is in line with what the US government and its allied Powers That be would want you to believe.

I also happen to know a dude at Twitter (nobody important I assume, just a developer, but still - an insider).

So I emailed the Twitter guy with the following message:

[...] I would really like to know why #wikileaks (as well as related hashtags) is not trending on Twitter, even though it obviously generates a massive amount of tweets, probably order of magnitude higher that anything else on Twitter.

At this point I still refuse to believe that Twitter is a part of a larger conspiracy to shut up WikiLeaks, although I am coming alarmingly close, having in mind PayPal and Amazon fiasco (I am in process of cancelling my accounts with both of these). I am still more inclined to think that you have a problem with your trending algorithm. But - if you do not get your act together, and soon, your credibility as a social medium is going to suffer a lot. After all, your only advantage is being the place where people post stuff (tech itself is quite trivial, wouldn't you agree?). If you lose credibility, you lose users and they may never come back.

So, if you know anybody on the trend analysis team (or whatever team takes care of this functionality at Twitter), I would be grateful if you passed my concerns to them.

Security advice for riding a motorcycle during a heat wave

2010-07-12T00:44:00.000-07:00

It seems like we are having a bit of a hot summer in Poland at last. Which mean riding a motorcycle becomes a bit of a challenge, because regular riding clothes become hot and uncomfortable. You basically have two choices:

lose the armour and ride in tshirt and shorts
stay in the armour and risk getting unconscious becouse of the heat

My advice is - absolutely do not ride without full armour! Reasons - the risk of injury while riding without it is grave. Your bare skin will be peeled off within a second from the contact with a tarmac. Also, during the heatwave, both you and the rest of the drivers on the street easily lose concentration, increasing the probability of accident

So, what to do? Armour, does get hot very quickly indeed. The solution is simple - get yourself a wet bandana and put it around your neck. It will keep you cool and fresh

Email from a headhunter and my reply

2010-06-28T10:33:00.000-07:00

I have received an email from a headhunter today and I have replied.

Read below:

Dear Gorycki

We have a CV on file from you which is more than a year old. Have you secured employment in Ireland or are you still interested in moving here? If so – can you email me an updated CV with details of your current remuneration package and outline what types of roles you are interested in hearing about? We will then update your file and keep you informed if suitable roles arise.

If you are no longer interested in hearing from us about new roles in Ireland – can you email us saying so and we can remove your file from our database.

Many thanks – and looking forward to hearing from you.

Kindest regards

Brian McFadden

Director

If this were the first time my last name is misused in an email from the headhunter, I woudl let it pass. But as these guys cannot be bothered to even try to think which name is the first name and which one is the last, I felt obliged to respond in the following way:

Dear McFadden,

Six years after Poland joined EU and since hordes of Poles invaded Ireland and the rest of British Islands, I really hoped to find you guys to be able to distinguish our first names from our last names. I am absolutely sure that there are numerous Januszes, Marcins, Piotrs, Wojteks and so on (hint - these are the first names of Polish folks, which I assume are still customarily used in your country as header lines in email and other correspondence).

On the other hand, Gorycki (typically mispronounced badly by everybody from England or Ireland), Nowak, Kowalski, etc. are our last names. I really expect professional headhunters from the Islands looking for potential candidates in countries like Poland to be able to tell which name is which and use the correct one while making the first contact. Or, if unsure, to at least try to figure out some sort of neutral form that would not be offensive - hint: using just the last name is very much a no-no. Or, if everything else fails, as the last resort, ask your secretary or a cleaning lady (she is more likely than not Polish, or Lithuanian, or Czech or Slovak - all of which will be able to help you out).

Kind Regards

Janusz Gorycki

Kalamon Software Web Site Facelift

2010-06-24T13:50:00.000-07:00

I have spruced up the KalamonSoft web site, to make it a bit less generic. What do you think of the new layout and colors?

DeskBeam Junior Coming Along Nicely. And Other Good Stuff

2010-06-02T12:13:00.000-07:00

I have been silent for a while, because I spend almost all of my evenings and nights coding my new baby - a lightweight screencasting application which I call DeskBeam Junior. Why "Junior"? Because this is just the first approximation of what I have in mind in the "Senior" version, which is going to come later, and which will include advanced video editing capabilities.

The picture on the left is the logo for the application, which I created a week ago. It is supposed to represent a flower. I hope you find it cute.

The whole thing at this moment is a bit secret/unstable/in statu nascendi, so I am not giving you samples, screenshots or downloads to try out just yet - but do expect some early access preview (of alpha-to-beta-ish quality) in a couple of weeks.

Unlike my previous creations, the software will not be open source - I am going to charge for it.

But - it will be rather cheap - to the tune of $20-$25, so I guess everybody should be able to afford it.

On a totally different note, I am preparing a long-ish post/presentation/essay on the subject of "Earned Value Measurements in Agile Projects" (yes, the title is deliberately controversial, as typical Agile approach is that EVM is bogus and worthless). I seem to have found an interesting metric for agile projects, which I guess nobody had though of before (at least Google does not know of anybody with an idea like that - I have been looking for it on the internets for a while). I hope the metric will be useful to some project managers out there, so if you are one of them, stay tuned. There will be formulas, graphics, and all sorts of good stuff

My yamaha scored a hit today :)

2010-05-12T01:09:00.000-07:00

Ok, I drew first blood on the motorcycle today. Well, not exactly blood, but there were (probably) some minor scratches - I have nicked some van's right mirror while passing it in the traffic jam. So, if you read this Mr. van dude (yeah, right) - I am officially very sorry for that. I will be more careful next time

XJ6 it is

2010-04-14T06:55:00.001-07:00

Ok, so I bought me a Yamaha XJ6. Ain't it nice?

Now I have to learn how to ride it without getting killed or maimed. That is going to be interesting

Dan Pink on motivation

2010-03-23T01:12:00.000-07:00

A must see video from TED Talks - Dan Pink talks about some surprising findings about motivation and incentives

Highly recommended

Kawasaki ER6n or Yamaha XJ-6

2010-03-20T14:43:00.000-07:00

Hard choices to make...

Which one is more cool, which one will rule? :)

Obligatory Introductory Post

2010-03-16T01:59:00.000-07:00

This is going to be a blog about everything and anything. I will try to talk about programming and software most of the time I guess, but you are likely to find posts about music, motorcycles, my family, politics, weird web sites and whatever else I feel like writing about.

Be warned.