Leveraging AI in Language Preservation Efforts
OCT 2023
PERSONAL PROJECT
Increasing the efficiency and lessening the burden of effort on language preservation researchers
problem
Language preservation efforts are tedious, time consuming, and lack modern software dedicated to their purpose.
Researchers still use pencil and paper to transcribe work, requiring them to not only take the time to write down every word accurately, but also to come back later and spend more time to analyze and pull insights from transcriptions. Accurately translating one story from Apache to English took roughly 40hrs for Dr. Veronica Tiller’s research team, not including the time it took to record the story in the first place (citation).
A modern Solution
Streamline data collection and analysis through realtime, AI powered voice to text parsing and transcription.
This lightweight solution not only allows researchers to utilize tech they already own and know how to use, reducing costs and learning curves, but also removes the burden of effort from researchers by automating the most time consuming part of the research process.
Details
Realtime AI parsing
The main and defining functionality of the app. The algorithm visually parses out what the interviewer and interviewee are saying, both for quality control and assurance that the session is being properly recorded.
Touch free voice commands
Users can prompt updates to datapoints by using certain phrases, such as "what do you mean by..." or "can you clarify..." to ask for clarification.
Dual microphone mode
This feature utilizes the microphones present on both ends of modern smartphones to better parse and differentiate between the interviewer and interviewee’s lines and can be toggled on or off.
Influences from mozart
Mozart's genius "table music" allowed musicians to play across from each other at a table using the same sheet of music as they read from opposite ends. In the same vein, the dual microphone feature allows the user to conduct interviews in a candid, natural manner, as if they were engaging in normal conversation.
Data table upload
Upon completion of a recording, parsed data is uploaded to a data table where researchers can directly begin analysis and/or update/modify data. In addition, pre-existing data can be uploaded to the platform to inform the MLM prior to recording sessions.
Reflections
This work opens up a whole world of possibilities ranging from drawing features to record and document handwriting styles and stroke order/direction, expanding to a desktop version of the app, and even providing user submission functionality to build public databases and crowdsource research material. In addition there are so many extended applications beyond just language preservation that this work can be applied to, including but not limited to: medical consultations, journalism, and media translation.