In the everyday world, users spend a lot of time using surfaces such as desktops and tables. Within the context of the Ambient Intelligence program at FORTH, new ways of interaction have been developed spanning a wide range of surfaces, from desktop environments, to restaurant tables, or child furniture. A central concept in these ways of interaction is the visual detection of fingertip touch and physical objects upon pertinent surfaces, without the invasive instrumentation of objects with sensors.