A research-through-design case study · FTP 2.0

WHO OWNSTHEMEANING?

A conversation about technology, care, interpretation and the ownership of human stories.

Research through designHuman–AI interactionInterpretive ownership
Researcher / Designer
M Lakshmi Supriya
251530740016 · MA’Exp, Year 2
Srishti Manipal Institute of Art, Design & Technology · MAHE Bengaluru
↗ GitHub · FTP2.0 branch↗ Public-facing FTP2.0 experience
text content are all original words. Case study website designed using Vercel
01 / The question

What if the parrot isn’t understanding us?

freeing the parrot is not a simple, nimble project that came out of an overthinking session, but something that I have always been sitting with.

“When a machine sounds like it understands you, what are you actually experiencing?”
my first year dealt with how we understand human care at the individual and the institutional level and how consequently, services are built to facilitate that care or in a hope to facilitate that care.
01 / Language

Language can imitate meaning.

what happens when data gets processed and what happens when this happens through the glorified filter of artificial intelligence?

02 / The tension

Make the tension tangible.

what happens to representation of people, their gender, culture and stories when everything is parsed through an algorithm that is coded using binary rules, and ‘if…then…else’?

02 / Evolution

From a metaphor to an encounter.

the first thing I did, post guidance from both arko and paul was to decide what interactions formed primary – input, parrot, data output.

01 / Input

The participant

A point of entry into the interaction.

02 / Response

The performance

The parrot responds and sustains the encounter.

03 / Reading

The reading

Signals are interpreted after the conversation.

04 / Boundary

The boundary

The encounter raises questions about what a system can claim to know.

03 / Research context

two contexts. one underlying question.

my transdisciplinary project at Chamba sort out to gather conversations from the field on how people perceived menstruation, menstrual hygiene and care and the underlying superficial reverence towards superstition and fearmongering of what happens when the menstruating body crossed the thresholds that were set to contain it?

Field research / Chamba

care, culture and the body

from there my query developed to understanding how taboos, narrowmindedness towards care informed the larger policies at the institutional level

Hospital CX / Chennai

care as an institution

and so I worked at a hospital in Chennai for a month to understand how does this happen and much to my surprise, care is prioritized based on who pays more, who has influence at both the societal level and the financial level, and how does the notion of care take shape for them.

interestingly however, both levels of research had a central spine which was data.
Question 01

what exactly qualifies as data was the first question I have been carrying forward and why?

Question 02

who assumes the role of taking decisions to qualify information as what kind of data and to whom this data should reach and by what means?

Question 03

what happens when data gets processed and what happens when this happens through the glorified filter of artificial intelligence?

Question 04

and at last, care and empathy or the last of it, when these redacted stories and identities of people become what is left of them?

04 / The need to build FTP

the personal is part of the research.

i usually get my concepts by looking at life experiences but interestingly, a significant proportion of them are also derived from tv series and movies and hence “Person of Interest” by Jonathan Nolan played an important role in driving my ideas for the concept of ftp.

Three supplied cinematic references—Her, Person of Interest, and The Prestige—reworked into a cohesive charcoal-and-crimson collage with duotone grading, halftone texture, grain, and scanline treatment.
HER (2013)directed by Spike Jonze · AFI CatalogPERSON OF INTEREST (2011–2016)created by Jonathan Nolan · TV Guide creditsTHE PRESTIGE (2006)directed by Christopher Nolan · D23 film reference

poster imagery sourced from the supplied reference images; colour grading and halftone treatment applied for this case study. links above provide film and series credit references.

i built the legacy version of ftp for my core studio ‘By any Media necessary’ led by Manuel Beltran as a sarcastic jibe on how people looked at an AI chat bot as an emotional source of support (“Her” movie; (2013)) and I read on why this was the scenario – why people relied on an AI system to be their emotional support and it all routed back to the questions I asked in my first year; what happens when people remove themselves from care and the idea of providing care?

to understand this further, I had to go back to my childhood and teenage days of social isolation and being treated differently amidst my peers and how when the AI boom happened, my own experiences of chatting with the AI and understanding how that seemed to give me the thought of it being an alternative towards paying for a therapist, except

an AI-led human conversation will tell you what you want to know, but a human-to-human-led conversation will reveal to you what you NEED to know.

perhaps that was what the idea around ftp was. knowing that the AI in its present form is disappointing and to make it work, would need require for a collective lobotomization of identities and knowledge.

FTP1.0 / What changed

weaknesses in the legacy build.

there were too many touchpoints for the user to fully engage in the installation which made ftp1.0 quite the conundrum.

during the pre-seminar break, I took some time on if an opportunity arose to enhance ftp1.0 what would be the changes I make? and the first thing I did, post guidance from both arko and paul was to decide what interactions formed primary – input, parrot, data output.

05 / The experience

Enter. Interact. Reflect.

the project has been designed with the UI inspired from mystic tech and a popular Christopher Nolan movie “The Prestige (2006) where the themes for ftp2.0 are deceptive, speculative and adversarial outlined on the core question of what happens when a machine performs understanding convincingly enough that a human begins to believe it understands them?

01 / Deceptive

the performance

challenges the perception that convincing machine behaviour necessarily means understanding.

02 / Speculative

the reading

explores possible futures of human–AI relationships and outsourced care.

03 / Adversarial

the boundary

introduces friction that unsettles assumptions about machine understanding.

Why the glitch?

when simulated attention falters.

the glitch was designed to be a matter of occasional trust breakages from the user’s end, to mirror the drift offs in human conversations during situations of interactions.

we, as humans tend to zone out during conversations which could potentially falter trust in the other person interacting with us until we recall where we last left, except the recall itself is a distinctly human process, shaped by our attention, memory, context and the effort we make to reconnect with the other person.

when a machine can convincingly simulate attention, what happens when that attention falters? and when it does, who is responsible for repairing the trust that was built in the first place?
06 / Building FTP2.0

tools are part of the process — not the author.

i learnt that building the frontend and backend separately was only the beginning. connecting the kimi-built interface to the python backend required me to work through api routing, data exchange and production connectivity.

Frontend
React, TypeScript, Vite
Backend
Python, Flask
AI interpretation
Google Gemini API
Session management
Python session coordinator and event store
Analysis
Custom Python modules for linguistic, temporal and Navarasa trajectory analysis
Testing
pytest
Hosting / deployment
Vercel frontend · Render backend
Database / infrastructure
Neon was part of the supporting infrastructure
Development tools
Kimi · GitHub Codespaces · v0 · GitHub
Integration

dynamic data and session coordination

i moved beyond static card displays to server-generated readings, while aligning frontend stages with backend session states, card selection and participant resonance.

Recovery

error handling

i learnt to account for failed api requests, session-start failures and broken interaction flows, ensuring that the participant could recover instead of reaching a dead end.

Production

deployment and end-to-end testing

i realised that a system working locally does not guarantee that it will work in production. i had to troubleshoot vercel builds, api rewrites and the connection between the frontend and render-hosted backend, while testing the complete participant journey.

Multimodal

more than text

i extended the system beyond text by working towards incorporating audio transcription and facial-expression evidence into the session pipeline, raising further questions about how different forms of human expression are interpreted by machines.

Kimi was a frontend development tool, not part of the application’s runtime stack. GitHub Codespaces supported cloud-based development, debugging and testing of the Python backend.

07 / What broke, what changed

documented errors. architectural impact. mitigation.

The documented issues fall into three categories: conceptual/analysis gaps, integration and lifecycle failures, and deployment/runtime errors. Some required architectural changes, while others were implementation bugs that prevented the existing architecture from running correctly.

Analysis and interpretation

Documented error or gapImpact on FTP2.0 architectureMitigation
Post-session reading was too deterministic.The Deep Reader risked producing predictable readings not sufficiently grounded in the participant's conversation.Expanded post-session analysis through temporal, linguistic and Navarasa trajectory synthesis.
Missing temporal trajectory analysis.The system could not adequately represent how interaction patterns changed across a session.Added TemporalTrajectorySynthesizer to capture turn indices, timestamps, response latency, inter-turn gaps, repetition and typing/pause signals.
Missing linguistic and Navarasa synthesis.The analysis layer lacked a structured account of linguistic patterns and emotional signals over time.Added linguistic and Navarasa trajectory analysis modules. A documented limitation remained: default Shanta labels could distort aggregates when no emotion was detected.
Insufficient separation between evidence and inference.The system risked presenting interpretations as directly observed facts.The audit identified a need for structured INFERRED claims, supporting evidence, confidence and alternatives. The report does not establish that every part was fully completed.
Static or weakly contextualised card themes.Readings could feel disconnected from the participant's actual conversation.Added conversation-theme extraction and dynamic territories in the backend, with a regression test.

Integration and session lifecycle

Documented errorImpact on architectureMitigation
Frontend and backend were not initially connected reliably.The interface could not consistently access authoritative backend APIs in production.Corrected API proxy/routing and connected the frontend to the Render-hosted backend.
Frontend relied on static card content or fallback behaviour.Displayed content could fail to reflect the backend's generated reading.Wired the frontend to dynamic deck and session APIs, making the server the source of participant-facing cards.
The API response stripped semantic_anchor.The frontend lost information required to present dynamically generated card themes correctly.Corrected the backend response and added a regression test.
Session-start failures could leave the participant stuck.The interaction could fail before the main experience began.Added entry-success handling and retry behaviour for initial session-start failures.
Frontend stages and backend session states needed alignment.The interface risked advancing independently of the backend lifecycle.Aligned participant-facing stages with backend lifecycle states and card-resonance validation.
Multimodal input did not always maintain lifecycle continuity.Audio and facial-expression evidence risked failing to carry through the session pipeline.Added multimodal continuity and session-archetype checkpointing, fixed lifecycle issues and expanded end-to-end QA.

Privacy, persistence and runtime

Documented issueImpact on architectureMitigation
Session persistence was not sufficiently constrained by participant consent.Session data could enter persistent storage without the intended consent boundary.Introduced consent-gated session persistence and a consent-controlled persistence lifecycle.
Raw dialogue risked being persisted in event writes.The data layer could retain more conversational content than necessary for ongoing event storage.Excluded raw dialogue from ongoing Neon event writes and added tests verifying raw Parrot turn events were not persisted.
Raw transcripts could appear in Memory Chest reveal snapshots.Sensitive conversational content could leak into persistent profile/reveal representation.Added transcript redaction to Memory Chest reveal snapshots.
Frontend TypeScript compilation failed.The frontend could not compile for deployment.Corrected type handling and an undefined-value issue; the frontend build subsequently passed.
Backend startup failures from f-string braces and newline handling.The Python backend could not start, making its API and dependent session functionality unavailable.Escaped literal JSON braces and corrected newline escaping.
Production API rewrites and deployment configuration were incorrect.A working frontend build did not guarantee production requests reached the intended backend.Corrected Vercel API rewrites, the Render backend target and frontend output-directory configuration.

from local functionality to production reliability.

from heuristic readings to structured analysis. from loosely connected components to a coordinated system. from implicit data handling to explicit boundaries.

08 / What building taught me

learning to prioritise under constraints.

perhaps this would be the most important realisation of all, since models like Google AI Studio ran out of credits quickly with me having to have spent 2K in total to save my project for extra credits + 1200/- for cursor start india plan.

So, I had to resort to continuing the last of my cursor credits and finishing the build with v0 by Vercel (by luck, I had $5 worth of monthly free credits) by prioritizing tasks based on the most credits to least credits used and subsequently the impact that the build would have on the overall UX and architecture of ftp2.0.

01 / Systems

integration is part of the design

building frontend and backend separately was only the beginning.

02 / Evidence

working does not mean knowing

automated tests can verify implementation behaviour; they do not prove that an interpretation is accurate or fair.

03 / Constraints

resource-aware decisions

limited credits made task prioritisation part of the development process.

The recorded test suite progressed from 205 to 218, 238, 280 and 313 tests across documented development phases. These counts demonstrate expanding automated coverage, not proof that all interpretive or ethical risks were resolved.

09 / The question remains

not a diagnosis. an invitation to interpret.

the system risked presenting interpretations as though they were directly observed facts, weakening the transparency of its profiling process.

when a system interprets our words, assigns meaning to our behaviour and constructs a reading of who we are, who gets to decide whether that reading is valid?
retaining humanity in the age of AI-driven systems: redefining governance and policies towards ownership.

FTP2.0 is a research-through-design prototype. Expanding automated coverage and building a functioning pipeline do not, on their own, validate psychological interpretations or establish participant impact.