Storytelling and Visualization

A transition refers to the process or a period of changing from one state or condition to another according to the Oxford English Dictionary. In the visualization literature, transitions may be the focus of visualization and include both dynamic and static which are alternatives of presenting visualization. Static visualizations are those that do not rely on animation. Transitions may be considered part of narrative storytelling. However, we designate the literature here in its own category to reflect the importance of transitions and to keep related literature on this topic together. Several research papers focus on the transitions in storytelling. This is why they are separated into a special group.

In this section, the visual designs of transitions is generally static. The authors focus on presenting the trend of data along timelines. Robertson et al. evaluate three approaches of using bubble charts and attempts to discover which one works best for presentation and analysis. Tanahashi and Ma presents a storyline visualization which consists of a series of lines, from left to right along the time-axis. Liu et al. design a storyline visualization system, StoryFlow, to generate an aesthetically pleasing and legible storyline visualization. Ferreira et al. propose a method of visualizing a large amount of taxi data consisting of both spatial and temporal dimensions.

Static Transitions for User-Directed and Interactive Storytelling

The literature in this subsection focuses on interactive user-driven transitions. The user creates static transitions interactively, i.e., using a process they have some control over (as opposed to automatically).

TaxiVis proposes a method of visualizing a large amount of taxi data consisting of both spatial and temporal dimensions. This approach examines trends over time as opposed to individual taxi trips, visualizing data from a day in length, up to a year. Seasonal events such as Thanksgiving and Christmas can be compared in a like-for-like fashion. See Figure 22.

Figure 22. The (top-left) image shows the trips rendered on the map. However the cluttered view can be reduced by employing a level-of-detail approach (top right) which takes a subsample based on the order in which the trips occurred. The (bottom-left) image shows a density heat map of the taxi trips whereas the (bottom-right) image averages out the data in each region to make a regional density heat map.

Time selection widget allows the user to change the time frame of the visualization. Maps server as the canvas for the visualization. A graph of the raw data with time plotted to the x-axis and frequency of taxi trips on the y-axis. To reduce clutter, a density heat map is used. This can either be as points on the map or averaging out the data within regions on the map.

Taxi behaviour is a popular focus of research. Among others, Veloso et al. explored patterns and trends in taxi ride data looking at the relationship between pick up and drop off points. Liao et al. developed a visual analytics system to error check GPS data streamed from taxis.

Static Transitions for Parallel Storytelling

In this category of literature, the static transitions are shown in parallel. In other words, many transitions can occur simultaneously. Robertson et al. define a trend in data as an observed general tendency. The most common way to see a trend in data is to plot a variable's change over time on a line chart or bar chart. If there is a general increase or decrease over time, this is perceived as a changing trend. Robertson et al. propose two alternatives to animated bubble charts for visualizing trends in multiple dimensions and describes a user study that evaluates the three approaches for both presentation and analysis. In conclusion, Robertson et al. state that traces and small multiples work best for analysis.

The gapminder trendalyzer uses a bubble chart to show four dimensions of data, life expectancy is mapped to the x-axis, infant mortality is mapped to the y-axis, population is mapped to bubble size and continent is mapped to color.

An alternative multi-dimensional trend visualization provides the user with the ability to select particular bubbles such that the animation shows a trace line for the selected bubble as it progresses. See Figure 23. In a small multiples visualization, countries can be clustered based on position, size, and location. They are further grouped by continent and ordered alphabetically within each group. Robertson et al. is based on earlier work by Tversky et al. and Baudisch et al. Previous work is limited to small data set sizes (200 samples or less). Their work focuses on presentation rather than analysis and relies on animation to show trends over time.

Figure 23. Robertson et al. show the trace lines of the graph animation. The traces visualization shows bubbles at all x and y locations throughout the time frame. This is a conversion of an animation into a static image.

Visual Storylines, by Chen et al. is designed to summarize video storylines in an image composition while preserving the style of the original videos. Chen et al. present a new visual storylines method to assist viewers in understanding important video contents by revealing essential information about video story units and their relationships. The first step of the algorithm is to extract the storylines from a video sequence by segmenting a video into multiple sets of shot sequences and determining their relationships. See Figure 24. The second step is to visualize a movie sequence in a new type of static visualization by using a multi-level visual storyline approach, which selects and synthesizes important story segments according to their relationships in a storyline.

Figure 24. Chen et al. presents video shot clustering algorithm combines both visual and audio features to generate a meaningful storyline.

Chen et al. is based on the work of video summarization and first clusters video shots according to both visual and audio data to form semantic video segments.

Storyline visualization is a technique that portrays the temporal dynamics of social interactions by projecting the timeline of the interaction onto an axis. Tanahashi and Ma present a storyline visualization which consists of a series of lines, from left to right along the time-axis, that converge and diverge in the course of their paths. Algorithm overview is shown in Figure 25. The layout is based on a set of horizontal slots that divide the screen space along the y-axis. Each of these slots has the capacity to accommodate blocks of interaction sessions as long as they do not overlap in time. Rearranging lines takes the slot-based layout of interaction sessions derived from a genome and determines the order of the line segments in each interaction session and its alignment in order to reduce unnecessary wiggles and crossovers. In order to prevent such misleading effects, it is critical for the layout computation to include the removal of unnecessary white space to determine the final layout. Tanahashi and Ma is based on the idea of XKCD's hand-drawn illusion "Movie Narrative Charts" and develops an algorithm for general storyline visualization.

Figure 25. Tanahashi and Ma present the overview algorithm of generating storyline visualizations.

Storyline visualizations, aim to illustrate the dynamic relationships between entities in a story. Liu et al. design a storyline visualization system, StoryFlow, to generate an aesthetically pleasing and legible storyline visualization. It supports real-time user interaction, hierarchical relationships among entities, and the rendering of a large number of entity lines. The layout pipeline consists of four steps: relationship tree generation, session/line ordering, session/line alignment, and layout compaction. In the first step, StoryFlow creates a set of dynamic relationship trees for different time frames, in which the relationship trees are used to order sessions and entity lines. Next, sessions/lines between successive time frames are aligned to maximize the number of straight lines in the layout. Finally, a quadratic optimization algorithm is performed to obtain a compact storyline layout. See Figure 26.

Figure 26. Comparison of King Lear using both methods of layout; (a) StoryFlow; (b) previous method by Tanahashi and Ma. The StoryFlow layout presented in this paper focuses on minimising white space and efficiently ordering the story lines to ensure the most concise visual representation of a story. Intersecting lines represent interaction between characters and major events in the story are labeled to add clarity to the visualization.

Liu et al. is based on previous work of Tanahashi et al. Liu et al. add support for real-time interaction, hierarchical relationships, and a large number of entity lines.