Ok, after some emails with Unikron, the solution I have come across is to only use circle colliders when colliding with the tilemap edges. Circle colliders seem to be the only reliable ones when colliding with edge2D. So I have disabled the polygon collider and "filled" the colliding object with circles until conforming the desired shape. It is not perfect, it is indeed annoying, but works fine. It makes me think of the Lumpy Space Princess in Adventure Time, but it kind of works... hehehe
It is faster than disabling TK2D tilemap edge optimizations, which I tried and worked ok, but it was not a great idea performance wise.
Just wanted to let this idea for the people that "hit the same wall"