Auditions For Project: Hell's Theatre
"You know, lately Mr. Dramaturg has been sneaking off somewhere after dance practice..."
"You sure that isn't just Reurent having some alone time with the Stage-Director, Anno...?"
"I heard it, Yuki-chan! The voice of a young girl that wasn't Henrietta...!"
Executive Summary: I embark on a mission to teach an AI to become an actress.
#HouseDelaroux, #AIVTuber, #HellsTheatre, #TechnoDramaturgy, #Dramaturgy, #LLMVtuber, #Ollama, #LocalAI, #AICharacterDesign, #VtuberDevelopment, #AICompanions, #OpenLLMVtuber
List Of Achievements TODAY (Not Tomorrow, Hoaeh!)
✅ Installed OpenLLM-Vtuber
✅ Bugfixed microphone, configured Deepseek models
✅ Identified areas for improvements
Today is the 20th of March, 2025. I've been thinking about expanding the dramaturgy side of the business to beyond the theatre houses in the region. After some discussion, I have been tasked by our Stage-Director to do work for... vtuber startups.
I am a Dramaturg, my regular job is to teach actresses how to act. While I am quite enmeshed with traditional theatre, I am not completely unaware what a vtuber is. (Miss Delulz smacks the top of my head)
A vtuber is simply an actress who wears a skinsuit.
As far as I am concerned, it is not impossible to teach a vtuber to act.
To test my theories, I shall now teach an AI to become a vtuber!
ENTER: OpenLLM-Vtuber
Thankfully, I am not trying to make Pinocchio from scratch. There is already a major project called OpenLLM-Vtuber that lets you run a basic 'AI Vtuber' setup, it even advertises itself as a 'open-source Neuro-sama'!
Things That Worked With OpenLLM-Vtuber
✅ Works Out of the Box
Unlike a lot of projects which have dependencies after dependencies, the documentation for OpenLLM-Vtuber works out of the box, and you just have to follow the instructions step-by-step. One big boon is that I already have most of the requirements installed already from my earlier work with AI (Python, etc), and the base models that make the project work are inside the .zip file.
Room for Improvement
🚀Microphone Settings
Going into the project, I noticed that the model wasn't picking up sound properly. This had to do with the voice gating settings, and how the AI detects voice.
OpenLLM-Vtuber (I'll call the project Tess temporarily from now on) has an option that lets the microphone mute itself after you speak, so you don't interrupt the AI. At the same time, you also need to turn on the function to unmute yourself after Tess speaks.
Adjusting the automatic speech recognition settings (ASR) helps a bit:
Speech Prob Threshold - The lower this number is, the easier your voice is picked up. Can be a problem if multiple people are talking at once. I set it at 40.
Negative Speech Threshold - The lower this number is, the easier it is for the AI to think you are done talking. This influences you stops and pauses so the AI knows when to reply. I put this a bit higher at 45 to prevent odd stops.
Redemption Frames - The lower this number is, the shorter the timeframe for false-stops is detected. So the AI will react faster. I put it at 30, so the AI has to hear a definite stop from me and not cut me off mid-sentence.
What's Next?
The basic setup has been completed: The AI-Vtuber exists. However, just having the basic model doesn't quite satisfy me, so I think I will work with Tess behind the curtains a little more before deciding. There are a couple of parameters I need to work on:
Model - Most thinking models are out because I'm running Ollama and Ollama includes thinking inside the token generation, so Tess runs a little slow for my taste.
Voice - Tess is currently speaking in the kind of sassy Kroni-esque voice, which I don't really like. I like evil women, so I should investigate what kind of voice options are there and see what to do from here.
Live2D - I should investigate the options for free Live2D models. There really shouldn't be that many available. Sadly it only works with Live2D.
??? - Live2D might be a huge barrier most people cannot cross. Since I have seen how this works, could a pngtuber version of this program exist? I dunno. It feels like that's what the desktop assistant version is for.
Curtain Call
There is quite a bit to consider! I should think about "how best to reduce response time" first, and see what happens from then on! currently the model is running a Deepseek8b model, which is a little slow because my computer is weak!
R.I, デラ・ルーの大導劇神
HouseDelaroux.com
250322
"LLM-based Vtuber setup"
"Ollama AI VTuber guide"
"Local AI companion project"
"VTuber AI persona design"
"Live2D AI VTuber test"
"OpenLLM-Vtuber tutorial"
