Designing Designing a Brainstorm Augmentation System

A system that supports brainstorming by displaying images based on voice and text recognition.

The Brainstorm Augmentation System, or BAS, is a system that can be used during brainstorms and provides the users with images related to their process. It searches for related images by filtering the nouns from post-it text and spoken comments. When the user writes down the title the system will provide 6 images, and when the user places a post-it the system will provide 4 images. The user can then either accept or alter the image result to better match his or her desires.


As can be seen above and to the left, BAS consists of a projector, a camera, a processing memory and five array microphones. The camera registers the location and motion of post-its, written text and movement. This is processed and a task is then relayed to the beamer to adapt the projection to the actions of the user. The array microphones can record speech which, by analyzing the direction of the sound input, is then connected to the related post-it when it is placed. An example of a brainstorm scenario can be seen above and to the right, in which the post-its are yellow and the projected images are blue.

BAS can be used during any brainstorm that makes use of post-its. The user suspends the device above a table with a paper cover and turns it on. An oval will appear on the table, asking the user to write a title with the text ‘write title here’. After the user has started writing a title, a button will appear to confirm the completion of the title. For this, the system will recognize a finger-press of the button to avoid accidentally pressing it.


After title confirmation, the system will process the written and spoken text and project 6 related images around the title area. The users will proceed to add post-its. With each post-it the system processes the written text on the post-it and the related spoken text to find 4 images, which are then projected around the post-it. If one of the users disagrees upon an image, this image can be replaced by manually swiping the single image away. This swiping motion is registered by the camera and after this input has been processed the beamer will project the image in such a way that it will move along with the user’s swiping motion and disappear when the movement ends (see below).


After the image has disappeared, it will automatically be replaced by a new image (see below).


During the brainstorm snapshots are created of the moment the title is finished as well as when a new post-it has been added. Through this, users are able to browse through their process afterwards and able to see how their brainstorm has progressed (see below).


State diagram

In the movie above all the system states are shown. This includes the way user actions impact those states and how the system actions influence the next actions of the user. For example: the system will show a button with the text ‘accept title’ as feedforward to inform the user of the possibility to perform the action of accepting the title and proceeding to the next state. The user input would be ‘pressing the button’, which would then cause the system to continue to the next state where the user can start placing post-its.