Lip sync issues

FeelingTerrible

Hey,

I have currently been working on lip syncing, I am struggling to get it working with Papagyo v1 and v2(beta). I am working on 2D Characters using the lip sync texture on Anima2D characters and using the latest AC.

I have gone through the forums and seems some people have had the same issue. It works fine with text to speech but when I try to run it from Papagyo method it find the file but the lips only switch to the default 0 frame.

The txt file for lipsyncing is :

MohoSwitch1

1 rest

19 rest

18 WQ

Everything seems to be setup properly but its just not working.

Pamala doesn't seem to be able to download from their site anymore and I am struggling to get Sapi working on my computer. Not really looking for purchasing an asset.

Any general Ideas?

Thanks,

Will

ChrisIceBox

If the "From Speech Text" option works, then it sounds like your character is set up properly, and the problem lies with the speech processing.

Do any messages appear in the Console?  How are you assigning your lipsync files?  Automatically via Resources folders, or manually by assigning them to each speech line.  If the former, try unchecking Auto-name speech files? in the Speech Manager and see if manually assigning a file to a line causes it to properly play.

The sample txt file looks OK, so assuming the files are being loaded properly then we can try inserting some Debug.Log statements into the code to try to work out what's going on.  The processing of Papagao lipsync files occurs in Dialog.cs, around line 613:

else if (_lipSyncMode == LipSyncMode.ReadPapagayoFile && textFile != null)

As an initial test, find the line:

if (papagoyoLineArray.Length == 2)

and insert the following directly above:

Debug.Log ("Total lines: " + papagoyoLines.Length + ", This line: " + papagoyoLine + ", Line array length: " + papagoyoLine.Length);

What's the Console output when a given line is supposed to play?

FeelingTerrible

Hey, thanks for the reply,

It looks like it is being picked up properly as it runs through the debug code:

Total lines: 58, This line: 1 rest, Line array length: 7

Total lines: 58, This line: 19 rest, Line array length: 8

Total lines: 58, This line: 18 WQ, Line array length: 6

Total lines: 58, This line: 21 AI, Line array length: 6

.....

and these are reflected in the phonemes editor too:

B/M/P/MBP/FV/rest

AY/AH/IH/EY/ER/AI/E

G/O/OO/OH/W/etc/WQ

SH/R/Z/SF/D/L/F/TN/K/N/NG/H/X

UH/EH/DH/AE/IY/U

The mouth changes to the default frame 0 but then never moves.

ChrisIceBox

I apologise: that should have read:

Debug.Log ("Total lines: " + papagoyoLines.Length + ", This line: " + papagoyoLine + ", Line array length: " + papagoyoLineArray.Length);

If the "Line array length" now appears as 2, then the lines should process correctly.

We'll need to do a bit more debugging.  Add the following debug statements above the following lines:

Debug.Log ("Begin search for: " + searchText);
foreach (string phoneme in KickStarter.speechManager.phonemes)


Debug.Log ("Let's try in " + phoneme);
string[] shapesArray = phoneme.ToLower ().Split ("/"[0]);


Debug.LogWarning ("Yes! Frame = " + frame);
lipSyncShapes.Add (new LipSyncShape (frame, timeIndex, KickStarter.speechManager.lipSyncSpeed, 24f));


Debug.LogWarning ("Could not find " + searchText);
lipSyncShapes.Add (new LipSyncShape (0, timeIndex, KickStarter.speechManager.lipSyncSpeed, 24f)); // was 240

FeelingTerrible

Seems to not find the phoneme even though they are all in the list. I have played around the editor with caps and that but that shouldn't matter anyway as its converted to lower.

Greatly appreciating the rapid help BTW,

thanks

Will

ChrisIceBox

Strange - indeed, the lower-case conversion should be making these all get found.

What version and platform of Unity?  I'll attempt to recreate this.

FeelingTerrible

Using Unity 2017.3.03f Personal and AC 1.60.7.

Amazing

Thank you

ChrisIceBox

What about the platform?

Can't find any problem so far.  It could be line-ending or format related.  Are there any hidden spaces at the end of the lines?  Try saving the lipsync file in a different format.

FeelingTerrible

I'm running on Windows. There are no spaces and I have even removed all unnecessary lines so it is just ones that is being found.

I have added quotation makes around the debug code and there is not spaces surrounding them.

I have the lipsync files exported from Papagyo and then manually changed from .dat to .txt files. 

If you had an available install file for Pamala I could try that and it would be greatly appreciated as the download link doesnt work on their site right now.

ChrisIceBox

The links on the page listed in the Manual work for me: http://users.monash.edu.au/~myless/catnap/pamela3/index.html#getit

(Direct download link: http://users.monash.edu.au/~myless/catnap/pamela3/Pamela_03_install.exe)

I would still like to get to the bottom of this though.  Could you PM me your Speech Manager asset file together with one of your Papagayo files?

FeelingTerrible

Hey, weirdly the download for that particular install file from their site does not download for me, not sure if it is just a England thing or something..

I will PM you the files to see if you can figure it out. thanks