Welcome to 17000 Islands, an interactive experiment in documentary image making, and for its creators, a process of challenges and milestones.
The 17000 Islands project (17K) revolves around audio-visual explorations of Indonesia’s Taman Mini Theme Park by filmmakers and project directors, Thomas Østbye and Edwin, producer Paramita Nath, with funding provided by the Norwegian Film Institute, and a large amount of technical and creative innovation by Helios Design Labs.
The physical park itself is a idealized replica of Indonesia’s 17 thousand islands and their diverse cultures. The digital experience recreates these islands through the use of short films and audio clips, and allows the viewer to remix the original footage into their own interpretation. This new film is then added back into the experience as a new “island”.
Here’s a quick snapshot view of two years of hard work, in the form of two integral design challenges the project posed for us and the milestones reached through the effort.
Build a powerful web-based timeline editing console that would allow a viewer to assemble and edit video footage, sound, text, subtitles, as well as record audio and do all of this though the old plain and simple web browser.
Milestones: New Procedures
On an interface design and programming level, we pushed useful toolsets like jQuery and WebAudio, and HTML5 video and canvas elements to places that were difficult to get to but allowed for drag and drop clip assembly, super fine-grain control of media in and out points, and real-time compositing.
On a backend functionality level (nerd alert!), 17k uses websockets, Node.js, and FFMPEG in combination with in-browser hardware access to allow for audio recording through a microphone input straight through to the web-server.
On a work flow level, the 17k editor is all about content creation, both online and offline. The final save functionality involves the conversion of ones completed piece into a downloadable file that can be imported into Final Cut Pro for conversion into a complete high-res version of what is now a film, thereby extending the reach of the documentary back into the theatre.
Make this all fun and engaging, and to encourage an audience to become participants and part of an informative documentary experience.
Milestones: New Visual Language.
17k has a visual and conceptual counterpoint to the linear, timeline-based experience outlined above in the form of a very non-linear, organic experience, loosely labelled the Island Builder. Powered by a simple but complex algorithm, the Voronoi equation, the Island Builder divides a space into a crystalline lattice structure, allowing us to break the video into shapes other than the rectilinear.
People can pull and push these image fragments around to form new shapes and connections. The process is abstract, but the math is such that while maintaining the sense of play and fluidity, all interaction with this Island Builder is actually creating a film edit that is reflected in the linear, timeline-based experience.
On a technical level, it was an achievement to fold the math into a programming structure that could make a connection with the super-nonlinearity of a voronoi equation with the strict linearity of a timeline editor.
On a design level it was an achievement to glue everything together: image sequencing, canvas drawing, cursor tracking, drag and drop, and above all, naturalize the experience with colour, shape and sound.
When the dust settles
17k is borne out of a set of challenges. And it will challenge its audience. The user experience it offers it deep and complex, 17K does many things above and beyond what is described here. Of interest to us and the film-makers, apart from the coolness of the final piece itself, 17k contains the seeds of many new ideas to work with and build on in the future.