2d face on a 3d model, possible to use 2DTK?

[esdot]

We have a 3d model, with a 2d face (https://i.gyazo.com/4dc7866c83295e8ec15171ddb1b1edb4.png) and would like to drive the facial animations using 2d toolkit. Is this possible? Primarily we want to use the animation tools for building and organizing the different keyframes of each animation, controlling number of loops, etc.

Basically we just want to use a skinned mesh renderer, instead of a tk2dSprite to display each frame of an animation.

[unikronsoftware]

It should be possible, though possibly a bit tricky. You can create a new derived class from tk2dBaseSprite, that will assign the correct material and figure out the correct subUv region for the sprite. Let me know if you'd like me to go into more detail. Some of the optimisations in tk2d will be working against you here...

ps. if you don't care about trimming it becomes considerably easier, if you turn off rotation it becomes easier yet. Some features will never be compatible (eg. dicing, but I can't see why ou'd want that here)

[esdot]

Ok cool, I'll try and take a look and see if I can figure it out. We're not looking to use any special features, no rotation, no dicing non of that. Just basic playback control, and looping support, driven from a single 4k texture sheet.

Any more detail you can provide would be excellent, even just the barebones/highlevel approach in pseudocode.
Cheers,

[unikronsoftware]

Ok, so what you'd need to do is

1. Create a sprite collection with the faces, and disable rotation and trimming (this will be necessary later, but disable it for now).

2. Create a class that derives from tk2dSpriteAnimator, and override the SetSprite function. This will then be attached to the meshrenderer that draws the face detail.

3. In the SetSprite function, you'll need to set the parameters on material for the renderer appropriately, instead of actually setting it on a sprite. This will set the complete texture to it.

Code: [Select]

var spr = spriteCollection.spriteDefinitions[spriteId];
renderer.material = spr.material;

4. Now we need to actually zone in the correct sprite. It'll need to be a combination of shaders + parameter setting. Assume your model covers the full extents of the sprite, you'll need to then find a uv offset and scale that will scale the appropriate part of the material to fit. You can figure out the UV extents by using spr.uvs[0] spr.uvs[3]. Those coordinates are in UV space, so you just need to figure out spr.uvs[0] = offset and scale spr.uvs[3] - spr.uvs[0]. You should use a normal unity shader for this, as the tk2d shaders dont support this.

5. You should now have animating face sprites that you can play and control like a sprite.

I can cover the other bits in more detail (like supporting trimming, rotations - this will give you considerably better atlas usage, but you might not need it and it is a lot more work)

[esdot]

NICE! Got it working, it even works with multiple atlases. Thanks for the help

I hardcoded our Tiling to .25f, since all our face sprites are 1024x1024.

The subclass was as simple as this:
public override void SetSprite(tk2dSpriteCollectionData spriteCollection, int spriteId) {
        var spr = spriteCollection.spriteDefinitions[spriteId];
        var renderer = GetComponent<SkinnedMeshRenderer>();
        renderer.material = spr.material;
        renderer.material.mainTextureOffset = spr.uvs[0];
    }

And if you have more than 16 frames, can enable "multiple atlases" and it works great.



There were a couple tweaks I had to make to the source to get everything to work properly.

1. To get the custom inspector for your subclass, set [CustomEditor(typeof(tk2dSpriteAnimator), true)] in tk2dSpriteAnimatorEditor.cs
2. To stop the sprite from disabling itself, comment out Line 117 in tk2dSpriteAnimator
3. Remove line 137 from tk2dSpriteAnimator to prevent runtime error log
4. To prevent runtime error in editor, change line 259 of tk2dSpriteAnimationClipEditor to preview.Draw(r, Animator.Sprite? Animator.Sprite.GetCurrentSpriteDef() : null);

That's about it. Thanks for the help!

Only one thing remaining as an issue is the packing time... it currently takes close to 30 seconds, to pack 17 1024 textures into 2 atlases, w/ no padding. Any chance this can be expedited?