How define audio ref to the Subject 1 - Ref2V
I am trying <Audio 1> : reference - the (S1) speaker follows <Audio 1>'s voice timbre and measured delivery without copying the original signal.
then (S1) says : [English] Hello MiniMaxAI .
Or ? maybe like this ? <Audio 1> : reference voice timbre and measured delivery.
(S4) is the <Audio 1> voice.
then (S1) says with (S4) : [English] Hello MiniMaxAI .
Its not clear!
help... help me!
I'm just responding to say that I don't think it works. I have been over the guide in every way possible, and the only reference audio that works for me is direct pass-through. I would love to be wrong and have someone explain how they did it (and not just say "read the guide").
I haven't been able to get it to work either and I've followed the prompt guide basically perfectly. It always just starts with gibberish or just plays the audio of the file itself until it may or may not interrupt it with my prompted dialogue.
Lipsyncing the referenced audio works perfectly but actually transforming the voice to make it say what you want without the original file being imposed into the scene seems impossible.
It works, but the comfy node passes the audio directly as well as copy the voice.
I can clone any voice but the audio goes all messed up trying to speak a garbled version of the input audio as well for any non speaking moments. You can force speaking throughout the video and it will work. But any silence becomes a garbled version of the input audio.
I think the comfy node is to blame tbh. The node messes up A LOT when trying to disconnect audio. It wont allow me to disconnect it until i refresh the page.
I actually got it to work now. Just gotta connect it to the ref_video_audio connections instead of ref_audio and prompt accordingly...
For example:
subject_definitions:
...
<Subject 1> ...
<Audio 1> is the voice-timbre reference for <Subject 1>, containing a spoken english vocal layer.
summary:
...
<Audio 1> is used as the voice-timbre reference for <Subject 1>.
retention_analysis:
...
<Audio 1>: reference - it's vocal timbre guides the spoken voice of <Subject 1> without copying the original signal.
and then prompt whatever the character is supposed to say in the full prompt:
<Subject 1> says : <d>[English] This finally seems to work. </d>
Edit: Seems to have been a lucky gen with the pitch and voice adjusting to whatever I needed but it doesn't seem to work that way. Still getting the gibberish/imposing when doing it how it's meant to be used. I guess it's a conditioning issue with the node as rocky533 said.