pshipkov
I spent a moment this morning to see if something can be done about baking arbitrary deformations to animated joints + skinCluster using SOuP nodes.
The results were pretty good and i figured i should post them here.

The red and blue overlapping objects shown below are the skinClustered representation and the original deforming meshes.

http://www.soup-dev.com/videos/deformations_to_skin_cluster.mov
Screenshot.png

http://www.soup-dev.com/videos/deformations_to_skin_cluster_1.mov
Screenshot-1.png 

Here is a little script that takes care of everything.
Select one or multiple mesh objects and run it.
Set the number of samples higher if the results are not satisfactory.
Depends on the object shape and type of deformations, the voxelBinding flag helps a lot or does not, so run a test with and without it to see what works best.

Code:

import maya.cmds as mc
def bakeDeformationsToSkinCluster(samples=100, voxelBinding=True):

  sf = int(mc.playbackOptions(q=True, min=True))
  ef = int(mc.playbackOptions(q=True, max=True))

  d = {}
  for n in mc.ls(sl=True, o=True):

    # non-intermediate mesh shape
    l = mc.listRelatives(n, pa=True, s=True, ni=True, typ="mesh") or []
    if len(l) == 0: continue

    # intermediate mesh shape
    io = None
    l = mc.listRelatives(n, pa=True, s=True, typ="mesh") or []
    for n2 in l:
      if mc.getAttr(n2+".io") == True:
        io = n2
        break
    if io == None: continue

    # create some SOuP nodes
    m2a = mc.createNode("mesh2arrays")
    mc.connectAttr(io+".worldMesh", m2a+".inputMesh")
    mc.setAttr(m2a+".type", 3)
    mc.setAttr(m2a+".count", samples)
    mc.setAttr(m2a+".relaxIterations", 10)
    pomi = mc.createNode("pointsOnMeshInfo")
    mc.connectAttr(io+".worldMesh", pomi+".inMesh")
    mc.connectAttr(m2a+".positionArray", pomi+".inPositionPP")
    mc.setAttr(pomi+".outputData", 1)
    mc.setAttr(pomi+".uvs", True)
    uv2pom = mc.createNode("uvsToPointsOnMesh")
    mc.connectAttr(m2a+".positionArray", uv2pom+".inGeometry")
    mc.connectAttr(n+".worldMesh", uv2pom+".inReferenceMesh")
    mc.connectAttr(pomi+".outUvPP", uv2pom+".inMapPP")
    pomi2 = mc.createNode("pointsOnMeshInfo")
    mc.connectAttr(n+".worldMesh", pomi2+".inMesh")
    mc.connectAttr(uv2pom+".outGeometry", pomi2+".inPositionPP")
    mc.setAttr(pomi2+".rotations", True)

    # create joints
    joints = []
    n2 = mc.duplicate(n)[0]
    n2 = mc.rename(n2, n+"_skinCluster")
    group = mc.createNode("transform", n=n+"_joints")
    p = mc.getAttr(uv2pom+".outGeometry")
    r = mc.getAttr(pomi2+".outRotationPP")
    for i in range(len(p)):
      j = mc.createNode("joint", p=group)
      mc.setAttr(j+".radius", 0.1)
      mc.setAttr(j+".t", p[i][0], p[i][1], p[i][2])
      mc.setAttr(j+".r", r[i][0], r[i][1], r[i][2])
      joints.append(j)

    # create skinCluster
    mc.select(n2, joints)
    sc = mc.skinCluster(bm=0, nw=True, wd=0, omi=False, dr=4, rui=True)
    if voxelBinding == True:
      try: mc.geomBind(sc)
      except: pass

    # remember some things
    d[n] = [m2a, pomi, uv2pom, pomi2, joints]

  # bake joints animation
  if len(d) > 0:
    mc.select(cl=True)
    panels = mc.lsUI(p=True)
    for mp in panels:
      try: mc.modelEditor(mp, e=True, vs=True)
      except: pass

    for i in range(sf, ef+1):
      mc.currentTime(i)
      for n in d.viewkeys():
        p = mc.getAttr(d[n][2]+".outGeometry")
        r = mc.getAttr(d[n][3]+".outRotationPP")
        for j in range(len(p)):
          mc.setAttr(d[n][4][j]+".t", p[j][0], p[j][1], p[j][2])
          mc.setAttr(d[n][4][j]+".r", r[j][0], r[j][1], r[j][2])
          mc.setKeyframe(d[n][4][j]+".t")
          mc.setKeyframe(d[n][4][j]+".r")

    for mp in panels:
      try: mc.modelEditor(mp, e=True, vs=False)
      except: pass

  # clean-up
  for n in d.viewkeys(): mc.delete(d[n][:-1])



Attached the torus scene here.
The other one with the torso is from the scenes archive (mush.ma).
Quote 0 0
-sam-
Hi,
i'am new to SOuP and to this forum, so first i want to thank you for making these tools available and to share you time and knowledge here.

I'am on maya 2015 and i can't get the same result as shown on the picture for the taurus. The joints are all created on the same area of the mesh instead of being "scattered" so the skinned mesh doesn't fit the deformation/animation at all and only a portion of the joints seems to have influence on verts. Even with increasing the joints numbers, switching voxelBinding flag doesn't make a difference. Do you have an idea about what's wrong here ?

Is it possible to make this setup work with a selection of existing joints?  I'm sure you see where i'am going, this kind of setup would be so great for game engine pipeline as shown by Hans Godard and Webber Huang.
Both of their works seems to rely a lot on the paper about Smooth Skinning Decomposition with Rigid Bones (SSDR), i don't have the math/science background to "evaluate" the difference between this method and what you bring as a solution with SOuP, but how do you compare the two?

Thank you.

Webber Huang :


Hans Godard :




Quote 0 0
pshipkov
Hi Sam,

recently in VR we needed to move some deforming geo to Unreal and because unreal does not have the nice alembic support CryEngine has, so i had to do the clumsy thing. With this said - i know very well where you are coming from :)

Can you download the latest SOuP version (just released it) and give it a try. This tool is built into it already with GUI and stuff.

I cannot promise, but will try to take a look soon at your other request about measuring "deltas" and "translating" that to skinWeights.

From the top of my head:
for each joint
    move the joint
    measure and store array of "deltas" for all points (as a scalar array of distances, but not the vectors)
    write the non-zero values as skinCluster weights for the joint in the loop

Then we run normalize skin weights and that should be it.
As a rule of thumb i don't read papers so i may be wrong on this one, but it should be easy to verify my assumption. :)
Quote 0 0
-sam-
Hi Peter,

Everything is working as expected with the latest release, thank you.

Thanks for considering my request, you'll make a lot of happy people i'm sure ;)

About the work of Hans Godard and Webber Huong based on the SSDR paper, it seems that their algorithm performs several iteration of calculation until they hit a certain error threshold indicated by the user, you can also indicate the Maximum Influence needed, this parameter is quite important depending of the game engine you are using.

What's your thought on VR ? Some major studios have been making the jump lately (ILM Experience Lab etc).

Cheers.
Quote 0 0
pshipkov

To be honest i find the whole idea of baking arbitrary deformations to skinCluster and set of joints a bit clunky. It just does not scale. If you know what i mean. The proper solution is to use Alembic caches in the game engines. CryEngine provides it out of the box and we are helping ourselves in Unreal.
With that said, i am a bit hesitant to spend more time on the idea. Maybe later. :)


I will restrain myself from commenting on VR. Only time will tell if the stars will line up for this one.
It looks promising right now.


Now, about baking of rig deformations to existing fk skeleton + skinCluster(s).
Check the code below. It seems to be holding pretty well here.

Code:

import maya.cmds as mc
import maya.mel as mm
import maya.OpenMaya as om

def bakeDefsToFkSkeleton(joint, nodes=None, maxInfluences=8, useRootJoint=True):

  if nodes == None: nodes = mc.ls(sl=True, o=True) or []
  if type(nodes) != list: nodes = [nodes]
  nodes = list(set(nodes))
  if len(nodes) == False: return

  # some buffers
  ids = {}
  joints = {}
  deltas = {}
  counts = {}
  neutrals = {}
  p = om.MPlug()

  # filter-out all non-mesh objects
  # store the neutral point positions for the rest
  for m in nodes:
    if mc.objExists(m+".worldMesh") == False:
      print("WARNING: Object should be of type mesh: "+m)
      continue
    if mc.nodeType(m) == "transform":
      m = mc.listRelatives(m, pa=True, s=True, ni=True, typ="mesh")[0]
    sl = om.MSelectionList()
    sl.add(m+".worldMesh")
    sl.getPlug(0, p)
    pa = om.MFloatPointArray()
    o = p.asMObject()
    om.MFnMesh(o).getPoints(pa, om.MSpace.kWorld)
    neutrals[m] = pa
    counts[m] = pa.length()
    deltas[m] = {}

  if len(deltas) == 0: return

  # remember FK skeleton hierarchy
  # unparent all joints
  l = mc.listRelatives(joint, ad=True, typ="joint") or []
  for j in l:
    joints[j] = mc.listRelatives(j, pa=True, p=True)[0]
    mc.parent(j, w=True)
  if useRootJoint == True: joints[joint] = None

  # move joints, measure point offsets, store distances
  for j in joints.viewkeys():
    ty = mc.getAttr(j+".ty")
    mc.setAttr(j+".ty", ty+1)
    for m in deltas.viewkeys():
      fa = om.MFloatArray(counts[m])
      sl = om.MSelectionList()
      sl.add(m+".worldMesh")
      sl.getPlug(0, p)
      pa = om.MFloatPointArray()
      pa2 = neutrals[m]
      o = p.asMObject()
      om.MFnMesh(o).getPoints(pa, om.MSpace.kWorld)
      for i in range(counts[m]): fa[i] = pa[i].distanceTo(pa2[i])
      deltas[m][j] = fa
    mc.setAttr(j+".ty", ty)

  # get rid of all deformers attached to the mesh objects
  mc.select(deltas.keys())
  mc.DeleteHistory()

  # apply skinCluster deformers to the mesh objects
  for m in deltas.viewkeys():
    mc.skinCluster(m, joints.keys(), bm=0, nw=True, wd=0, omi=False, dr=4, rui=True)

  # rebuild the FK skeleton
  for j in joints.viewkeys():
    if j != joint: mc.parent(j, joints[j])

  # lookup table for skinCluster influence ids
  sc = mm.eval('findRelatedSkinCluster("'+deltas.keys()[0]+'")')
  l = mc.listAttr(sc+".matrix", m=True) or []
  for a in l: ids[mc.listConnections(sc+"."+a, s=True, d=False)[0]] = a.split("[")[-1][:-1]

  # bake stored distances as skinCluster weights
  for m in deltas.viewkeys():
    sc = mm.eval('findRelatedSkinCluster("'+m+'")')
    for j in joints.viewkeys():
      fa = deltas[m][j]
      for i in range(counts[m]):
        mc.setAttr(sc+".weightList["+str(i)+"].weights["+ids[j]+"]", fa[i])
    mc.setAttr(sc+".maxInfluences", maxInfluences)
    mc.setAttr(sc+".maintainMaxInfluences", True)
    mm.eval('select '+m+'; doNormalizeWeightsArgList 1 {"4"}; removeUnusedInfluences()')

Quote 0 0
-sam-
Hi Peter,

your code wasn't working for me out of the box, and i'am not that used to the api but i've manage to  have something working by making some changes. I'm not using MPlug() but getting the shape and working from there.
I've made my test on the mush file exemple, and the result are 'ok' but not perfect, some part like the elbow or the second joint of the spine are still funky but overall it's seems to be on the good path.
You were certainly using MPlug and the worldmesh attribute for a good reason and my changes may have left something out of the solution.

Alembic is without a doubt a plus for games engines, i like how they used it in Ryse to bake "destruction" etc. But for things like facial setup, LODS,  the way a solution like SSDR offers like baking blendshape to joints seems to be holding things up pretty well as joints are way cheaper in memory than alembic no ?

The result maya file is too big for being attached, sorry.
For those who will test the code, don't freak out ;) it takes around 15 seconds to perform the task.(on the mush file exemple).

edit : there's two joints on the skeleton that have their translate attributes locked, unlock them before using the function.


regards.

Code:

import maya.cmds as mc
import maya.mel as mm
import maya.OpenMaya as om

def main(joint, nodes=None, maxInfluences=8):
    if nodes is None:
        nodes = mc.ls(sl=True, o=True) or []
    if type(nodes) != list:
        nodes = [nodes]
    nodes = list(set(nodes))
    if len(nodes) is False:
        return

    # some buffers
    ids = {}
    joints = {}
    deltas = {}
    counts = {}
    neutrals = {}

    # filter-out all non-mesh objects
    # store the neutral point positions for the rest
    for m in nodes:
        if mc.objExists(m+".worldMesh") is False:
            print("WARNING: Object should be of type mesh: "+m)
            continue
        sl = om.MSelectionList()
        sl.add(m)
        dag = om.MDagPath()
        cp = om.MObject()
        sl.getDagPath(0, dag, cp)
        # Extend to the shape node.
        dag.extendToShapeDirectlyBelow(0)
        shape = dag.node()
        if shape.apiType() == om.MFn.kMesh:
            mfn_mesh = om.MFnMesh(dag)
        pa = om.MFloatPointArray()
        mfn_mesh.getPoints(pa, om.MSpace.kWorld)
        neutrals[m] = pa
        counts[m] = pa.length()
        deltas[m] = {}

    if len(deltas) == 0:
        return

    # remember FK skeleton hieararchy
    # unparent all joints
    l = mc.listRelatives(joint, ad=True, typ="joint") or []
    for j in l:
        joints[j] = mc.listRelatives(j, pa=True, p=True)[0]
        mc.parent(j, w=True)

    # move joints, measure point offsets, store distances
    for j in joints.viewkeys():
        ty = mc.getAttr(j+".ty")
        mc.setAttr(j+".ty", ty+1)
        for m in deltas.viewkeys():
            fa = om.MFloatArray(counts[m])
            pa = om.MFloatPointArray()
            mfn_mesh.getPoints(pa, om.MSpace.kWorld)
            for i in range(counts[m]):
                fa[i] = pa[i].distanceTo(neutrals[m][i])
            deltas[m][j] = fa
        mc.setAttr(j+".ty", ty)

    # get rid of all deformers attached to the mesh objects
    mc.select(deltas.keys())
    mc.DeleteHistory()

    # apply skinCluster deformers to the mesh objects
    for m in deltas.viewkeys():
        mc.skinCluster(m, joints.keys(), bm=0, nw=True, wd=0, omi=False, dr=4, rui=True)
    # rebuild the FK skeleton
    for j in joints.viewkeys():
        if j != joint:
            mc.parent(j, joints[j])

    # lookup table for skinCluster influence ids
    sc = mm.eval('findRelatedSkinCluster("'+deltas.keys()[0]+'")')
    l = mc.listAttr(sc+".matrix", m=True) or []
    for a in l:
        ids[mc.listConnections(sc+"."+a, s=True, d=False)[0]] = a.split("[")[-1][:-1]

    # bake stored point distances as skinCluster weights
    for m in deltas.viewkeys():
        sc = mm.eval('findRelatedSkinCluster("'+m+'")')
        for j in joints.viewkeys():
            fa = deltas[m][j]
            for i in range(counts[m]):
                mc.setAttr(sc+".weightList["+str(i)+"].weights["+ids[j]+"]", fa[i])
        mc.setAttr(sc+".maxInfluences", maxInfluences)
        mc.setAttr(sc+".maintainMaxInfluences", True)
        mm.eval('select '+m+'; doNormalizeWeightsArgList 1 {"4"}; removeUnusedInfluences()')


Quote 0 0
pshipkov
Hi Sam,

it is strange that the original code was not working for you. Can you provide a Maya scene that shows the problem.



Are you guys not using blend shapes for facial animation ?
I didn't notice a performance difference between blendShapes and joints+skinCluster for facial setup with 120 poses, for example.
Alembic is only for rbds and soft body deformations here.



There will always be a difference when approximating multiple deformers by baking them down to a skinCluster. There is no way around it. If you have a scene where you see significant differences after baking that don't match the "original" setup please post it here and i will take a look.
Quote 0 0
-sam-
Hi Peter,

in this post i've attached the result file using your original code and in the next post the one with my modified code. I used the mush.ma file provided in the exemple scene for soup archive.

When i was saying that your code wasn't working i meant that it wasn't giving a correct result, it's only the result of a smooth binding operation. I'am on maya 2015.
The problem seems to be in those few lines as the neutral dict doesn't populate and the pa.lenght is 0:

Code:

    sl.getPlug(0, p)
    pa = om.MFloatPointArray()
    om.MFnMesh(p.asMObject()).getPoints(pa, om.MSpace.kWorld)
    neutrals[m] = pa
    counts[m] = pa.length()


And i've also needed to remove this line to have the parenting/unparenting of joints working:
Code:

joints[joint] = None


Thank you.
Quote 0 0
-sam-
And now the result file with the modified code, i've manage to make it smaller.

Quote 0 0
pshipkov
Ok simply put - in Maya2016 and later we can do this:
om.MFnMesh(p.asMObject()).getPoints(pa, om.MSpace.kWorld)
but in previous versions it fails, so we have to do this instead:
o = p.asMObject()
om
.MFnMesh(o).getPoints(pa, om.MSpace.kWorld)
So, i changed my original code. Give it a try.

Quote 0 0
djx
This is really interesting stuff. I'm following along, but maya becomes unresponsive when I run your code on the mush example scene (1 processor ticking along).

I've noticed in my own scripts that this type of thing...
mc.setAttr(sc+".weightList["+str(i)+"].weights["+ids[j]+"]", fa[i])
... runs incredibly slow in 2016 SP1.

So I'm interested to hear, are you using 2016 SP1 yet Peter? 
Quote 0 0
-sam-
Hi Peter,

It's working with maya 2015, but i still needed to remove the line 46:
Code:

joints[joint] = None


Thanks.
Quote 0 0
pshipkov
I added a keyword argument to the function's signature in my code to accommodate for that.

@djx
I don't use Maya2016 SP1.
How many points your are testing with ?
Here it takes just a moment with 3800 and then another test with 18000.
Quote 0 0
djx
Yeah, SP1 has broken setAttr performance (I can only say for sure on windows). Setting skinCluster weightList on about 1500 verts with 5 joints takes a fraction of a second in 2015 and 2016 (gold) but it takes about 90 seconds on 2016 SP1.

David
Quote 0 0
pshipkov
Dang !

Will try to replace this with a simple api call.
Quote 0 0

Add a Website Forum to your website.