Sprite textures get destroyed during run time?

[grit]

I have a problem with sprite collections being destroyed in a scene, at least it seems that way, in combination with another plugin "LiveTexture" by prime[31]. However, I think the problem is on this side, so I try to find a solution here first. The problem occurs on my iOS devices in release mode – not in debug mode, though.

More details – sorry if the description gets quite long, but I want to make sure, you understand what is happening. (Also I am just a beginner in Unity):

I have a rather simple 2D scene with some tk2d sprites, all sharing one sprite collection – and a plane for showing a cutscene first in front of the whole scene. Via scripting I load a video texture (a video file) into the material of the plane when the scene starts. (All that is done using that other prime[31] plugin.)

As soon as the video finishes, I hide the video plane – and here comes the problem: In release mode (development build turned off) all the tk2d sprites now have the video texture or parts of it applied to them – as if the sprite collection, they are supposed to use, is not available any more. This does not occur, when I build for development. I guess, this is because the sprite collection is not unloaded then. (?)

However, I played around a bit, to find out, what's going on. I want to share three observations:
1) The problem does not appear with Unity Sprites. They all look fine, but I just switched to tk2d, because of the better 2D handling and more flexible asset management.
2) If I add a Unity Sprite to the scene and assign the atlas to it, that the other tk2d sprites are using as well, the problem does not appear any more. I  assume, the Unity Sprite prevents now the sprite collection from being destroyed. (This is my actual work around right now, but it is obviously not how it should be solved.)
3) If I add a tk2d Sprite with a tk2d Sprite Animation to the scene – also using the same shared sprite collection as all the others – I see that the sprite collection is gone for a wink after the video is done, and comes back when the animation jumps to the next frame, as if it retrieves the sprite collection back into the scene.

PS: I know, I should always try to limit the assets. The scenes I have right now are really small, sometimes just a handful of sprites using a 1024 x 2048 atlas alltogether. The video files range from about 5 MB to sometimes 25MB, but the problem appears with all of them. The video texture is loaded from the StreamingAssets folder btw, in case this is of importance.

Thank you in advance for help and suggestions.

[unikronsoftware]

We dont do any asset management - basically tk2d sprites store references to objects, and this gets loaded automatically by unity when the sprite is loaded and unloaded when the objects that reference it are destroyed. If Unity is messing about with things it shouldn't be (and I've seen this happen...), the behaviour will be very unpredictable.

Based on what you're saying, it sounds very much like the material is getting stomped on somehow, but you will need to verify it. Write a bit of debug code using OnGUI() that will get a sprite in the scene, then get the sharedMaterial.mainTexture. Does it look wrong when drawn into a GUI.DrawTexture? If it looks OK there, then it implies the material is what is broken. That will explain #3.

In any case if you cant figure it out you can write a script to rebuild all sprites after your video is played, and this will "fix" the issue, but really, this should not be happening. Its worth spending a little bit of time working it out.

[grit]

Thank you for the quick answer. I did what you suggested and displayed the texture via GUI.DrawTexture on the screen. It looks totally fine – and at the same time the tk2d sprites look fine as well. But only when I do this. Is it possible that the GUI rectangle now holds the texture in memory? Or do you have another idea, what is happening?

Also, could you give me a hint, please, how I can rebuild all sprites by scripting?

Kind regards.

[unikronsoftware]

Ok. It looks like the materials have somehow been corrupted. This could be a unity bug... I have seen something like this happen in another project which didnt use tk2d... You can rebuild sprites by doing something like this -
tk2dSprite[] allSprites = Object.FindObjectsOfType(typeof(tk2dSprite)) as tk2dSprite[];
and for each of them, call ForceBuild().
Hopefully that will fix it.

[grit]

Thank you for the quick reply. Unfortunately, the ForceBuild() call does not fix the issue.
I double-checked, if my tk2dSprites are really collected into the allSprites array. Here my code snippet (I use JS, but that should not make any difference, should it?):

      var allSprites : tk2dSprite[] = FindObjectsOfType(typeof(tk2dSprite)) as tk2dSprite[];
      Debug.Log("Please repair: " + allSprites.length);
      for (var spr : tk2dSprite in allSprites) {
         spr.ForceBuild();
         Debug.Log("This is sprite called: " + spr.CurrentSprite.name);
      }

More suggestions are highly appreciated.

[unikronsoftware]

Take one of the broken materials and put it on a normal unity plane. Is it broken on there too? If the material is bust, you can probably convince it to fix itself somehow.

[grit]

I added a plane with the atlas material of the sprites to the scene, as you suggested, and put it in front of the movie plane.
While the movie is playing, the material of that plane looks as it switches all the time very fast between the sprite material I put on it and the movie material! (However, I logged the name of the assigned texture in Update() and it keeps saying it is the right one.)

Also I just had the idea of shifting the movie plane to the side so I could actually see what happens to the tk2dSprites behind the movie while it is playing: The sprites look fine all the time, just until I stop the movie / the movie finishes: Then they have the movie texture on them, as described before.

I have no idea what is going on here.
As a workaround I can put a Unity sprite somewhere hidden or use the GUI.DrawTexture idea somewhere outside the screen. But still, this is not very elegant.

[unikronsoftware]

Right. So the texture ref on the material is being broken. I don't know how the prime31 plugin works, but I can think of a couple ways this could break. I'd speak to Mike from Prime31 to see if he has any ideas.

[grit]

Thank you so much! 
It is very handy, that you know him.
I am lookng forward to hear what you find out.
Kind regards.

[unikronsoftware]

Hey mike replied on twitter, are you applying this to a sprite?
https://twitter.com/unikronsoftware/status/484306250583531520

[grit]

Hmm, all my tk2d sprites in the scene use the material, that was generated by creating the sprite collection atlas, called "atlas0 material.mat" (see pic1).
The video plane has its own material, called "video.mat" (see pic2).

However, I don't understand following: When I choose "find references in scene" in the context menu of the video.mat, the results also include my navi buttons, which are just tk2d buttons using the same material as the other tk2d sprites in the scene (see pic3). The video plane should be the only object in the scene using the video material. Why do the buttons show up there as a search result?

...
Because I found this weird now, I just did another test: I deleted the navi buttons from the screen – et voilà: The sprites look fine now after the video finishes. Also I did the test with displaying the unity plane again and now it looks totally fine.

Looks like, my navi buttons do something bad here. Do I overlook something? Can there be a problem, because the sprites are instances of prefabs? (The navi button prefab refers to yet another material.)
Or is it even possible, that there happened something weird by reassigning materials now and then when I tested a lot of stuff before? Maybe it helps to build the navi buttons from scratch again? I will give it a shot now...

[unikronsoftware]

I have no idea what could've gone wrong, but it looks like you're getting somewhere with this, which can only be good news

[grit]

Ah! I think I was guessing too quickly: I built the navi from scratch again (tkUIItems like the control prefabs in your package >> Basic >> BasicButton).
Same happens.
(Just learned the "find reference" thing is not weird, but just because the button's target is the tk2dCam which refers to the video plane. Ha.)
I noticed, if I play the video to the end (have no other choice when I don't have navi buttons any more ) it does not happen (or rarely happens?). When I interrupt the film by using my forwards button, it happens (more likely?). I wonder why, both events (film finishes and pressing forwards) trigger the same function – this would be a question for Mike.
Also unfortunately concerning the unity plane test: I was stupid: When I did that first test some days ago, I assigned another (a third) material by mistake to that plane. That one shows the flickering (don't understand why is that, but maybe not of interest now?). When I assign the material used by the other tk2d sprites as well, it looks fine with or without old or new navi buttons. Sorry for the confusion.

Maybe a problem is also the memory? I am testing on my iPad 1 and I get a lot of memory warnings.

[grit]

... I just made a new build, with the scene reconstructed as it was before – without any "workaround" – and now I see no problem with the materials.
I am confused and I guess you, as well. Sorry.
I think I should have a fresh look at it again later to proceed more analytically.
Argh.

[grit]

It is me again.

I just couldn't stop the investigations and now noticed something!

Actually it seems the navi buttons HAVE something to do with the problem, but in another way as I thought of:
Because what just happened was: By "reconstructing" my scene I made a mistake and assigned the button textures from the wrong atlas to the navi buttons, means they now didn't use the same material as the other sprites of the scene any more. Now after the video finished or was interrupted the sprites all looked correct. The same is the case, when I don't have navi buttons at all. (The two navi buttons sit in front of everything all the time: during the video is playing and afterwards in front of the other sprites as well.)

Is it possible, that in the moment, the video is done or interrupted = texture killed, that the system sweeps all the stuff from memory, that is just visible in front of the video plane, and thus destroys the material of the navi buttons? Well, this would probably be a question for prime[31].

I still wonder, why this should be the case. Also why this is not happening, when there is a unity sprite with the relevant texture hanging around somewhere "behind" the scene.

Another thing I observed: The materials get indeed not messed up that often, when I let the video finish itself instead of interrupting it. I should address this to prime as well.
Still everything quite obscure to me.