At the heart of Djali is an exploration into counterfactual and
imaginative reasoning about possible worlds and lives, mediated by
experimentation in both narrative structure and technology. The exhibit
integrates advances in computer vision, computer graphics, automated
reasoning, and artificial intelligence generally, with innovation into
the structure of storytelling itself.

As a visitor moves through the exhibit, they encounter a structure that
embodies Djali, a ruminating artificial intelligence. Djali detects
visitors through an array of sensors, and communicates with them via
sound and visuals on a large display that covers much of its surface.

Narratives

Djali takes the visitor through a sequence of vignettes, short stories
that serve as windows into imaginary worlds and the lives within them,
loosely linked together by a backdrop that touches on questions of race,
class, technology and the climate. Each window offers a glimpse into a
different counterfactual branch, presenting stories of worlds that might
exist in the future, that could have existed but didn’t, or that could
never exist in reality. Djali guides visitors through these different
worlds and encourages them to explore and reflect on the choices and
circumstances that created and continue to shape the world we inhabit.

Mediums

Djali employs an array of storytelling mediums, both traditional and
new. Traditional static imagery, video, and sound come together to
depict the detailed environments and narratives of each hypothetical
world.

Beyond these established technologies, Djali uses “Figments”, a new
technology developed for the exhibit, to give visitors a novel sense of
presence within the alternative realities. The Figments, enabled by
composing very recent advances in computer vision, computer graphics,
and artificial intelligence, create a viewing experience goes beyond the
sum of its parts. Experientially, as a visitor shifts their perspective
by moving their head or gaze, the computer reconstructs the displayed
image correspondingly, creating the perception that the visitor is
genuinely looking into an alternative world through a window, or portal.

Interactions

Visitors are active participants within the exhibit. Their presence and
movements guide Djali through the counterfactual branches, giving them
an influential role in shaping the narratives. As visitors engage with
the windows and explore the presented alternative realities, they are
encouraged to reflect on the choices, consequences, and possibilities
that exist within each world.

The Venice Biennale Context

The 2023 Venice Biennale of Architecture, titled "The Laboratory of the
Future," provides an ideal setting for Djali. The Biennale challenges
architects, designers, and artists to push the boundaries of their
disciplines, exploring new technologies and ideas that have the
potential to reshape the future.

Djali aligns with this vision, embodying a spirit of innovation and
exploration. By merging the technical and creative talents of Basis and
GKZ, the exhibition offers a unique perspective on the potential futures
that lie before us.

Intended Impact

The intended impact of Djali is multifaceted, aiming to challenge
visitors' assumptions about the real world and its properties. By
presenting alternative realities, the exhibit encourages visitors to
reflect on which aspects of our world are immutable, as opposed to those
that are coincidental or subject to change.

Djali seeks to showcase the richness of everyday lives in scenarios
where roles are altered and hierarchies are upended, while also
prompting visitors to question the limitations of their imagination. By
mediating these worlds through artificial intelligence, the exhibit
raises questions about the creative potential of artificial
intelligences, and the degree to which these cognitive faculties can or
will remain distinctly human.

Furthermore, the exhibition honors the role of Jali, also known as
Griot, the traditional West African storyteller responsible for
preserving and sharing cultural knowledge and wisdom. By incorporating
the concept of Jali, which shares its root with "Djali," into the
exhibition, the creators emphasize the significance of storytelling in
promoting empathy, understanding, and connection across diverse cultures
and societies.

Ultimately, Djali aims to transcend the boundaries of conventional
storytelling by using artificial intelligence as a medium to deliver
imaginative narratives that inspire visitors to contemplate the
possibilities of our shared future and the role each individual plays in
shaping it.

System at a Glance

The Djali architecture and its creation process consist of several
components:

1. The physical structure: Visitors to the exhibit encounter a
   physical structure—a large display within an enclosure. Equipped
   with high-speed cameras, microphones, and embedded high-performance
   computers, the structure can sense its environment and detect the
   presence of people.

2. Narrative: The structure embodies an observant, archival, and
   ruminating artificial intelligence named Djali. Djali presents its
   thoughts, both real and imaginary, through various forms of
   expression.

3. Library of works: Djali conveys the narrative by sequencing
   together various pieces of work.

4. Photography: Images and visual content form an integral part of
   the narrative and the visitor’s experience.

5. Figments: The most unique experiences are presented through
   Figments. For the visitor, viewing a Figment feels like looking
   through a window or portal into another environment frozen in time.
   Unlike conventional videos or images on a screen, a Figment
   dynamically changes in response to the visitor’s viewpoint, creating
   the sensation of peering through a window. Constructing this
   experience requires (i) capturing 3D representations of scenes from
   images, (ii) tracking the viewpoint of the visitor, and (iii)
   real-time rendering of the 3D scene on the display with the correct
   perspective relative to the estimated viewer’s viewpoint.

6. Interactive compositions: Visitors actively engage with the
   exhibit, influencing the narratives and experiences presented to
   them.

7. The AI Architecture: The sensing, rendering, composition, and
   interactivity are all mediated by a unified artificial intelligence
   architecture. In other words, Djali functions as an artificial
   intelligence both within the narrative and in actuality.

Physical Construction

The physical structure of the Djali exhibition is composed of
extruded anodized aluminum, providing a robust and functional framework.
At the center, the lenticular display is placed, while the GPU computers
are concealed at the back.

The frame features multiple panels, filled with translucent
polycarbonate sheets. The structure also supports the sensing
infrastructure, with cameras, microphones, and speakers fixed to the
frame, ensuring seamless integration of technology and design.

AI Architecture

The AI architecture for the Djali exhibition consists of
interconnected software modules. Input data from cameras and microphones
is processed by a GPU-powered machine running these modules.

The gaze estimation module uses computer vision techniques to track
the visitor’s irises and triangulate their gaze in 3D space. The gaze
prediction module predicts the visitor’s gaze several milliseconds
into the future, ensuring 3D models are rendered at the appropriate
perspective.

Cognitive components, such as text-to-speech, speech-to-text,
and the NERF 3D rendering module, are read from and written to by
the cognition core, which serves as the central controlling module,
deciding what to present to the visitor.

The audio source separation module processes audio streams,
isolating the visitor’s voice from background noise. The asset
library stores 3D NERF models, videos, still images, and other media
assets for use in the exhibit.

Finally, the data sinks, including the lenticular display and
speakers for positional audio, present visual and auditory elements of
alternative realities to the visitor, mediated by the cognitive core.

Figments

Figments are a unique and central aspect of the Djali exhibit, providing
visitors with a snapshot into alternative worlds. The experience is made
possible by a combination of off-axis projection and Neural Radiance
Fields (NeRF), which capture high-fidelity scenes in 3D. This
cutting-edge technology enables the Figments to offer an immersive and
captivating glimpse into the richly detailed environments of each
hypothetical world.

As visitors engage with the Figments, they are guided by audio vignettes
and, which in some cases, are narrated by Djali itself. This
multi-sensory experience fosters a deeper understanding and connection
to the alternative realities presented.

Narrative

In a world reshaped by revolution and technology, old borders and
nations have faded, replaced by a new order of interconnected
city-states governed by benevolent artificial intelligences. In this
realm, idealized collectivism triumphs over individualism, while the
Earth’s ecosystems are revered as a sentient and sacred deity.
Citizenship, no longer bound by physical borders, is granted and
maintained through constant surveillance and adherence to neo-religious
ecological beliefs.

Within the densely populated urban polities, digital citizenship grants
individuals access to communal resources, generous basic income, and the
freedom to traverse allied city-states. These urban hubs, meticulously
designed to minimize humanity’s impact on the Earth, house ninety
percent of the world’s population within a land area no larger than the
Tokyo metro area.

Despite the system’s efficiency, a predictable number of individuals
find themselves denied or stripped of their citizenship. Cast out from
the city-states, they are left with no choice but to seek refuge in the
forgotten relics of the old world, joining the unrehabilitated
criminals, capitalist ideologues, and religious zealots. In these
chaotic, class-driven principalities, the remaining ten percent of the
global population resides, living under the rule of warlords who thrive
on cyber warfare, terrorism, and contraband trafficking within the
city-states.

This is a world divided by ideology, where the future of humanity
teeters between the pursuit of Earth preservation and the dangerous
allure of unchecked ambition. It is a tale of revolution, technology,
and the complexities of a society shaped by artificial intelligence,
ecological reverence, and the remnants of the old world order.

References and Concept Work

![image](/djali/images/wistful.png

Contributors

Research Development: Zenna Tavares, Ria Das, Shaiyan Keshvari, Karen Schroder, Emily Mackevicius, Wyatt Garfield, Michelle Yi

Creative Development: Zenna Tavares, Kibwe Tavares, Gaika Tavares, Lullyn Tavares, Jessica Au, Ying Suen

Article: Karen Schroeder, Zenna Tavares

Acknowledgments: APOC Store, Zuckerman Institute, The Venice Biennale of Architecture, Jessica Wimbart