Papagayo Lip-Sync Not Working

calebvetter

Been trying to get this working and it looks like everything should work, but it doesn't for some reason.

Phonemes are set up and working correctly:

And when I select "From Speech Text" it works just fine, but when I select "Read Papagayo File" it just shows the first frame of my talk animation during the dialog line.

I added Debug.Log("Added frame: " + frame + " at time: " + timeIndex); to the Dialog.cs file inside the _lipSyncMode == LipSyncMode.ReadPapagayoFile if statement, as well as Debug.Log("Added frame: " + frame + " at time: " + i); inside the _lipSyncMode == LipSyncMode.FromSpeechText if statement.

Here's the debug output when I have "From Speech Text" selected:

And this is when I have "Read Papagayo File" selected:

I also thought maybe it was a frame rate issue, so I tried changing the "Process speed" but still nothing.

Is there something I'm missing?

ChrisIceBox

Going by your logs, the frames are correct - just the time may be a factor out.

If only the first frame is showing, I'd guess it's processing too slow (unless it's processing so fast only the last frame is visible). Try adjusting the "24" in the Papagayo block's LipSyncShape creation.

calebvetter

Yep, that was it. Looks like there might be a problem with the math in the declaration of LipSyncShape at the bottom of the Dialog.cs file? Papagayo does generate 24 fps files, but the function isn't really accounting for it.

I changed:
public LipSyncShape (int _frame, float _timeIndex, float speed, float fps = 1f)
to:
public LipSyncShape (int _frame, float _timeIndex, float speed, float fps = 15f)
so the default is 15fps (which is what it looks like the "From Speech Text" runs on)

And also changed:
timeIndex = (_timeIndex / 15f / speed / fps) + Time.time;
to:
timeIndex = (_timeIndex / fps * speed) + Time.time;

Now it works perfectly.

ChrisIceBox

Thanks for the feedback, I'll look into it.