Tip of the tongue phenomenon

Feature idea - what if Calico used speech recognition to map words that were said while something was being drawn? It wouldn't have to be accurate in the least. If only a few words can be captured, it would key people in to what was being discussed. For example, after a discussion is done, you could use a lasso that would circle a set of figures, and a list of words that were mentioned while those were being drawn appeared in a transient floating box. It wouldn't necessarily be sentences, just key words. The speech recognition software could have a high error rate, even 50 percent miss, but it would still be extremely useful. It would need to be calibrated, test if the gap would need to be only while it's being drawn, or if it needs everything said 1 minute before and after. It could also be calibrated to pick out "important" words, or all words. This could be done retroactively by assigning a time stamp to all drawn objects, and a time stamp to all words recorded, then map them to each other afterwards.

What makes this useful and possible is the tip of the tongue phenomenon. People are much better at recognition than they are at recall. While reviewing a session, the words would be similar enough that they would instigate a recognition of events. A psych study was done on tip of the tongue events, and people recited items which were either close phonetically or semantically. So, even if the voice recognition software completely butchered the word, it would be close enough that it clue the reviewer to an idea that was at the tip of their tongue.

I would like to perform a study eventually using Calico and this concept. The study would involve two sessions, one session using the board, and one session a few days later that reviewed the first session to allow an incubation process for ideas. The latter test would involve questions reviewing the first drawing session, having them look at key words, and measure how many new ideas are spawned by looking at the old. Also, a search feature would send the user to elements drawn that included that key word.

I don't want to develop this feature myself, but if another person joins the team, I'd like have them work on this.

Reference 1
Reference 2
Reference 3

Multitouch up and Running!

After the hardware had a few misadventures around the world, I finally got the multitouch setup up and running. I'm using the multitouch technologies based off the guide put out by the NUI Group, and had my hardware mostly made by Harry van der Veen (I owe them a lot of thanks!)

My setup is pretty crude, but it worked for a good test run. The acrylic screen is only 15" inches, but I was pretty surprised at how close I could get the projector. My home projector only needed to be about 3 1/2 feet away.

Here's the setup I was working with.

I had the screen setup vertically, but the fingers didn't give it enough pressure for my FTIR setup. I had to press my fingers a little harder than it was comfortable, but I think gravity would make this much easier once the screen is horizontal.

The hardware responded a bit slow once I got it running. You can see a few points were missed in the picture below.
I'm using a camera that advertises 25 fps, so maybe a faster camera will give better results. Maybe if I tried some software that used point interpolation it would react better. As it was though, the recognition was pretty slow. (Btw, the hanging projection paper gives the image the ripple effect, another reason to go to a horizontal setup =D ).

Overall though, the project is looking good. The touchlib framework acts as a web server, so the multitouch is compatible with any programming language. You just need to connect to the proper ports (3000/3333), and any C++, Java, or Flash application can work with it. Technically you can interact with other computers across the network, but if the reaction time is less than a fraction of a second it'll feel wrong.

Next steps are to build a cage and look into getting a faster IR camera.

java this statement

There's a peculiar situation that I got myself into, and wanted to keep a note for myself here. When you've got an inner class, and you want to refer to the outter class using the this statement, you reach it by using ParentClass.this

Example:

public class MyClass {

public void someFunction() {
...
}

private class InnerClass {
public void function() {
MyClass.this.someFunction(); // <-- important line here!
}
}
}

Feature: Autoexpand for scraps

When I am writing something, and the mark finishes just outside the scrap I'm writing in, the scrap will gain a pimple to encompass what I just wrote.

Implementation: Collect all of the points in the last stroke, find the maximum bounds in all four points, and draw an ellipse that encompasses that region. Must be minimal, must look natural.

Electronic paper surfaces

Something odd I've just noticed, the physical consistency of my 19" AMW LCD screen is the same as paper, while my tablet PC's screen is as slick as glass. A complaint that I've heard (and share) is that tablets don't feel like paper, so it's awkward to write on them. If only there was a very thin sheet of clear paper that I could apply over the tablet PC to make it rough, I'd be much more willing to use the tablet PC for every day note use.

I've also read about the problem with consistency in several papers. I wonder what if there are any rough plastic surfaces I could attach to my tablet screen...

Question to self

Does more notes equal more creativity?
Does encouraging more sketching equate to more creativity?

Enough Free Space to Feel Creative Freedom

Just a quick thought that came to mind while reading a DENIM publication. A feeling I get when am working with a small monitor is feeling cramped. Does this space vary from person to person? Is there an optimal monitor size and resolution (x/y) that feels good?

I shove an object until it's at my peripheral, and then my main work is centered on my center of vision (fovea). Exactly how much space do I need to not feel cramped. Does the noise that exist in my peripheral vision affect my ability to give attention to the center of vision?

Note to self, update later when I have time with research on the above questions.

Update: I was reading Francois's (from Stanford) thesis, and he wrote that one group preferred not to use LiveBoard (a digital tool) because it couldn't fit all of their ideas successfully. They said a proper tool should include at least 100 different ideas.