Lip syncing issue
Hi!
I'm working with 2d and Unity sprites.
I got my talking animation working and I wanted to apply lip syncing to it (just text for now).
I don't know if I'm doing something wrong, I am doing everything according to the manual and tutorial, but I can't seem to get lip syncing to work. I set up the phonemes but it didn't look right (it was choosing the wrong frames though they were set up correctly both in game editor and animation clip).
I did some testing and I set up 4 frames for "phonemes" A, B, C, D and I made the player say ABCD over and over just to see if it matched and it doesn't. So basically I have
Frame 0: A
Frame 1: B
Frame 2: C
Frame 3: D
But when the player speaks, it just animates ABC. For D (frame 3), it just repeats frame 0. I tried playing around with it but I couldn't get it to animate the sequence properly and it was a bit erratic, I couldn't really see what was the error. Perhaps when I tested and created / deleted frames and phonemes, something was misconfigured? Or maybe this parses actual English phonemes from the text instead of single letters?
Any ideas?
Thanks in advance.
Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Comments
An update on this: I think I managed to solve this with a workaround. I added an extra empty frame in the phoneme inspector, even though I technically shouldn't. So, I got a 4 frame lip animation, but in the phoneme inspector I have 5 frames (0, 1, 2, 3, 4), the last one being empty. This way the all phonemes show the exact mouth shape.
Strange. I shall attempt a recreation.
What's your AC version, and can you share a shot of your Speech Manager settings?
I'm using AC 1.71.3
Screenshots: https://www.dropbox.com/sh/lll068zhyr7jz7d/AAC1e6CKkhMNIWBVkKHkc3I2a?dl=0
And your animation engine is set to "Sprites Unity"?
This animation mode handles lipsyncing by playing the talk animation at a normalized time, so e.g. frame 2 of 5 will be 40% along the animation. You'll need to make sure that the talk animation itself has the same number of frames as you have phonemes.
Got it, I will add an extra frame to the animation. And yes, the animation engine is set to Sprites unity.
Hi Chris, just an update on this. We moved the project to 3d and we are not dealing with the talking animations just yet, so that's why there were no news... but I have another little project in 2d with AC 1.71.8 and I have my player all set up with its animations and everything, and I ran into the same thing as last time when dealing with lipsyncing in 2d via dialogue text: even though there is the same number of frames in the animation as in the speech settings, and all of them are properly assigned to their letters, when you play the game, you never get the last frame. In my case this corresponds to F sounds. So I went ahead and did what I shouldn't do as last time: I added an extra frame in the speech settings... and just like last time, it worked that way (even though it shouldn't, as you explained).
In case you want to try to reproduce it:
I have a 6 (0 - 5) frame animation and then I assign them to phonemes (letters, in this case) via the speech manager. My last frame is for the F sound (in Spanish there is no V sound), so I have a frame just for that one. If I leave it like this, the F frame (#5) will never show up, even when I just try to make it say "FFFFFFFF"--it will just default to frame #0(an Ah sound in my case). But... if I add an extra frame in the speech manager (so: 0 to 6), that F shows up when required, as well as all the others.
So technically I don't have a "problem" right now, it's just that the accidental work around I found shouldn't work, as you have explained.
Just in case, I am using Sprites unity as my animation method.
Let me know if this is clear and/or you need more info or clarification.
Thank you in advance.
Edit: Recreated.
At least on my end, the issue is that playing the last frame (due to a normalized time value of 1) will cause the animation to play the first frame if set to loop.
Viewing the animation asset's Inspector and unchecking Loop Time should prevent this.
Great! It works on my end as well. Thanks again!