At the heart of Djali is an exploration into counterfactual and imaginative reasoning about possible worlds and lives, mediated by experimentation in both narrative structure and technology. The exhibit integrates advances in computer vision, computer graphics, automated reasoning, and artificial intelligence generally, with innovation into the structure of storytelling itself.

As a visitor moves through the exhibit, they encounter a structure that embodies Djali, a ruminating artificial intelligence. Djali detects visitors through an array of sensors, and communicates with them via sound and visuals on a large display that covers much of its surface.

Narratives

Djali takes the visitor through a sequence of vignettes, short stories that serve as windows into imaginary worlds and the lives within them, loosely linked together by a backdrop that touches on questions of race, class, technology and the climate. Each window offers a glimpse into a different counterfactual branch, presenting stories of worlds that might exist in the future, that could have existed but didn’t, or that could never exist in reality. Djali guides visitors through these different worlds and encourages them to explore and reflect on the choices and circumstances that created and continue to shape the world we inhabit.

Mediums

Djali employs an array of storytelling mediums, both traditional and new. Traditional static imagery, video, and sound come together to depict the detailed environments and narratives of each hypothetical world.

Beyond these established technologies, Djali uses “Figments”, a new technology developed for the exhibit, to give visitors a novel sense of presence within the alternative realities. The Figments, enabled by composing very recent advances in computer vision, computer graphics, and artificial intelligence, create a viewing experience goes beyond the sum of its parts. Experientially, as a visitor shifts their perspective by moving their head or gaze, the computer reconstructs the displayed image correspondingly, creating the perception that the visitor is genuinely looking into an alternative world through a window, or portal.

Interactions

Visitors are active participants within the exhibit. Their presence and movements guide Djali through the counterfactual branches, giving them an influential role in shaping the narratives. As visitors engage with the windows and explore the presented alternative realities, they are encouraged to reflect on the choices, consequences, and possibilities that exist within each world.

The Venice Biennale Context

The 2023 Venice Biennale of Architecture, titled "The Laboratory of the Future," provides an ideal setting for Djali. The Biennale challenges architects, designers, and artists to push the boundaries of their disciplines, exploring new technologies and ideas that have the potential to reshape the future.

Djali aligns with this vision, embodying a spirit of innovation and exploration. By merging the technical and creative talents of Basis and GKZ, the exhibition offers a unique perspective on the potential futures that lie before us.

Intended Impact

The intended impact of Djali is multifaceted, aiming to challenge visitors’ assumptions about the real world and its properties. By presenting alternative realities, the exhibit encourages visitors to reflect on which aspects of our world are immutable, as opposed to those that are coincidental or subject to change.

Djali seeks to showcase the richness of everyday lives in scenarios where roles are altered and hierarchies are upended, while also prompting visitors to question the limitations of their imagination. By mediating these worlds through artificial intelligence, the exhibit raises questions about the creative potential of artificial intelligences, and the degree to which these cognitive faculties can or will remain distinctly human.

Furthermore, the exhibition honors the role of Jali, also known as Griot, the traditional West African storyteller responsible for preserving and sharing cultural knowledge and wisdom. By incorporating the concept of Jali, which shares its root with "Djali," into the exhibition, the creators emphasize the significance of storytelling in promoting empathy, understanding, and connection across diverse cultures and societies.

Ultimately, Djali aims to transcend the boundaries of conventional storytelling by using artificial intelligence as a medium to deliver imaginative narratives that inspire visitors to contemplate the possibilities of our shared future and the role each individual plays in shaping it.

System at a Glance

The Djali architecture and its creation process consist of several components:

1. The physical structure: Visitors to the exhibit encounter a physical structure—a large display within an enclosure. Equipped with high-speed cameras, microphones, and embedded high-performance computers, the structure can sense its environment and detect the presence of people.

2. Narrative: The structure embodies an observant, archival, and ruminating artificial intelligence named Djali. Djali presents its thoughts, both real and imaginary, through various forms of expression.

3. Library of works: Djali conveys the narrative by sequencing together various pieces of work.

4. Photography: Images and visual content form an integral part of the narrative and the visitor’s experience.

5. Figments: The most unique experiences are presented through Figments. For the visitor, viewing a Figment feels like looking through a window or portal into another environment frozen in time. Unlike conventional videos or images on a screen, a Figment dynamically changes in response to the visitor’s viewpoint, creating the sensation of peering through a window. Constructing this experience requires (i) capturing 3D representations of scenes from images, (ii) tracking the viewpoint of the visitor, and (iii) real-time rendering of the 3D scene on the display with the correct perspective relative to the estimated viewer’s viewpoint.

6. Interactive compositions: Visitors actively engage with the exhibit, influencing the narratives and experiences presented to them.

7. The AI Architecture: The sensing, rendering, composition, and interactivity are all mediated by a unified artificial intelligence architecture. In other words, Djali functions as an artificial intelligence both within the narrative and in actuality.

Physical Construction

The physical structure of the Djali exhibition is composed of extruded anodized aluminum, providing a robust and functional framework. At the center, the lenticular display is placed, while the GPU computers are concealed at the back.

The frame features multiple panels, filled with translucent polycarbonate sheets. The structure also supports the sensing infrastructure, with cameras, microphones, and speakers fixed to the frame, ensuring seamless integration of technology and design.

AI Architecture

The AI architecture for the Djali exhibition consists of interconnected software modules. Input data from cameras and microphones is processed by a GPU-powered machine running these modules.

The gaze estimation module uses computer vision techniques to track the visitor’s irises and triangulate their gaze in 3D space. The gaze prediction module predicts the visitor’s gaze several milliseconds into the future, ensuring 3D models are rendered at the appropriate perspective.

Cognitive components, such as text-to-speech, speech-to-text, and the NERF 3D rendering module, are read from and written to by the cognition core, which serves as the central controlling module, deciding what to present to the visitor.

The audio source separation module processes audio streams, isolating the visitor’s voice from background noise. The asset library stores 3D NERF models, videos, still images, and other media assets for use in the exhibit.

Finally, the data sinks, including the lenticular display and speakers for positional audio, present visual and auditory elements of alternative realities to the visitor, mediated by the cognitive core.

Figments

Figments are a unique and central aspect of the Djali exhibit, providing visitors with a snapshot into alternative worlds. The experience is made possible by a combination of off-axis projection and Neural Radiance Fields (NeRF), which capture high-fidelity scenes in 3D. This cutting-edge technology enables the Figments to offer an immersive and captivating glimpse into the richly detailed environments of each hypothetical world.

As visitors engage with the Figments, they are guided by audio vignettes and, which in some cases, are narrated by Djali itself. This multi-sensory experience fosters a deeper understanding and connection to the alternative realities presented.

Narrative

In a world reshaped by revolution and technology, old borders and nations have faded, replaced by a new order of interconnected city-states governed by benevolent artificial intelligences. In this realm, idealized collectivism triumphs over individualism, while the Earth’s ecosystems are revered as a sentient and sacred deity. Citizenship, no longer bound by physical borders, is granted and maintained through constant surveillance and adherence to neo-religious ecological beliefs.

Within the densely populated urban polities, digital citizenship grants individuals access to communal resources, generous basic income, and the freedom to traverse allied city-states. These urban hubs, meticulously designed to minimize humanity’s impact on the Earth, house ninety percent of the world’s population within a land area no larger than the Tokyo metro area.

Despite the system’s efficiency, a predictable number of individuals find themselves denied or stripped of their citizenship. Cast out from the city-states, they are left with no choice but to seek refuge in the forgotten relics of the old world, joining the unrehabilitated criminals, capitalist ideologues, and religious zealots. In these chaotic, class-driven principalities, the remaining ten percent of the global population resides, living under the rule of warlords who thrive on cyber warfare, terrorism, and contraband trafficking within the city-states.

This is a world divided by ideology, where the future of humanity teeters between the pursuit of Earth preservation and the dangerous allure of unchecked ambition. It is a tale of revolution, technology, and the complexities of a society shaped by artificial intelligence, ecological reverence, and the remnants of the old world order.

References and Concept Work

![image](/djali/images/wistful.png

Contributors

Research Development: Zenna Tavares, Ria Das, Shaiyan Keshvari, Karen Schroder, Emily Mackevicius, Wyatt Garfield, Michelle Yi

Creative Development: Zenna Tavares, Kibwe Tavares, Gaika Tavares, Lullyn Tavares, Jessica Au, Ying Suen

Article: Karen Schroeder, Zenna Tavares

Acknowledgments: APOC Store, Zuckerman Institute, The Venice Biennale of Architecture, Jessica Wimbart