Home Technical Talk
The BRAWL² Tournament Challenge has been announced!

It starts May 12, and ends Sept 12. Let's see what you got!

https://polycount.com/discussion/237047/the-brawl²-tournament

3ds Max - Directional Lightmaps maxscript, matrix/normal basis issues

Hi, I've been writing a maxscript tool to render out directional lightmaps using 3ds Max and while I have the process down the final result isn't quite right. The script rotates the surface normals of the mesh before rendering each of the 3 lightmaps and the lightmaps themselves look correct but when combined back together in the shader the light looks like its coming from the wrong directions.

The 3 lightmaps for an unwrapped box end up coming out like this:
Box01_dlm1.jpgBox01_dlm2.jpgBox01_dlm3.jpg

But the final lightmap shader outputs this:
ProblemDLM.jpg

I feel like there is a matrix transform i'm missing with the normal basis but i'm not sure.

Max doesn't exactly provide the greatest control over surface normals, tangents and bitangents so I've had to take a function from someone on cgsociety to generate the face bi/tangents manually and use those bi/tangents to transform the normal basis.

The code for skewing the normals is as follows:
  1. -----------------------------------------------------------------------------------------------------
  2. -- computeTangentSpace function taken from HalfVector on cgsociety
  3. -- The face position has been removed from the resulting matrices
  4. -- http://forums.cgsociety.org/showpost.php?p=3881243&postcount=11
  5. function computeTangentSpace obj =
  6. (
  7. local theMesh = snapshotAsMesh obj
  8. local tSpace = #()
  9. -- Do we have to flip faces?
  10. local flip = false
  11. local indices = #(1, 2, 3)
  12. if dot (cross obj.transform.row1 obj.transform.row2) obj.transform.row3 <= 0 do (
  13. indices[2] = 3
  14. indices[3] = 2
  15. flip = true
  16. )
  17. for nFace = 1 to theMesh.numFaces do (
  18. local face = getFace theMesh nFace
  19. local tface = getTVFace theMesh nFace
  20. local v1 = getVert theMesh face[indices[1]]
  21. local v2 = getVert theMesh face[indices[2]]
  22. local v3 = getVert theMesh face[indices[3]]
  23. local uv1 = getTVert theMesh tface[indices[1]]
  24. local uv2 = getTVert theMesh tface[indices[2]]
  25. local uv3 = getTVert theMesh tface[indices[3]]
  26. local dV1 = v1 - v2
  27. local dV2 = v1 - v3
  28. local dUV1 = uv1 - uv2
  29. local dUV2 = uv1 - uv3
  30. local area = dUV1.x * dUV2.y - dUV1.y * dUV2.x
  31. local sign = if area < 0 then -1 else 1
  32. local tangent = [0,0,1]
  33. tangent.x = dV1.x * dUV2.y - dUV1.y * dV2.x
  34. tangent.y = dV1.y * dUV2.y - dUV1.y * dV2.y
  35. tangent.z = dV1.z * dUV2.y - dUV1.y * dV2.z
  36. tangent = (normalize tangent) * sign
  37. local normal = normalize (getFaceNormal theMesh nFace)
  38. if flip do normal = -normal
  39. local binormal = (normalize (cross normal tangent)) * sign
  40. append tSpace (Matrix3 tangent binormal normal [0,0,0])
  41. )
  42. delete theMesh
  43. return tSpace
  44. )
  45. function SetNormalsModifier nodeList &editNormalsMod lightmapType =
  46. (
  47. normalBasis = [0, 0, 1]
  48. if (lightmapType == 1) then
  49. (
  50. return 0
  51. )
  52. else if (lightmapType == 2) then
  53. (
  54. normalBasis = [sqrt (2.0 / 3.0), 0, 1.0 / (sqrt 3.0)]
  55. )
  56. else if (lightmapType == 3) then
  57. (
  58. normalBasis = [-(1.0 / (sqrt 6.0)), 1.0f / (sqrt 2.0), 1.0f / (sqrt 3.0)]
  59. )
  60. else if (lightmapType == 4) then
  61. (
  62. normalBasis = [-(1.0 / (sqrt 6.0)), -(1.0f / (sqrt 2.0)), 1.0f / (sqrt 3.0)]
  63. )
  64. for j = 1 to nodeList.count do
  65. (
  66. obj = nodeList[j]
  67. -- Get the face TBN's
  68. meshFaceTBNs = computeTangentSpace obj
  69. normalFaceCount = editNormalsMod.getNumFaces node:obj
  70. normalVertexCount = editNormalsMod.getNumNormals node:obj
  71. -- Create an array of arrays to contain the vertex normals for averaging later
  72. normalsListArray = #()
  73. for i = 1 to normalVertexCount do
  74. (
  75. append normalsListArray #()
  76. )
  77. for i = 1 to normalFaceCount do
  78. (
  79. faceSelectionArray = #{i}
  80. normalSelection = #{}
  81. -- Get the normal indices of the selected face
  82. editNormalsMod.ConvertFaceSelection &faceSelectionArray &normalSelection node:obj
  83. -- Get the face TBN
  84. TBN = meshFaceTBNs[i]
  85. for k in normalSelection do
  86. (
  87. -- Transform the normal basis into tangent space?
  88. normal = normalize(normalBasis * TBN)
  89. -- Append the new normal to a list for this vertex
  90. append normalsListArray[k] normal
  91. )
  92. )
  93. for i = 1 to normalVertexCount do
  94. (
  95. normal = [0, 0, 0]
  96. -- Average the normals from the list for this vertex
  97. for k = 1 to normalsListArray[i].count do
  98. (
  99. normal += normalsListArray[i][k]
  100. )
  101. normal /= normalsListArray[i].count
  102. -- Set the vertex normal
  103. editNormalsMod.SetNormal i (normalize normal) node:obj
  104. )
  105. )
  106. )

The full script and a DirectX shader for you to look at is available here.

The script is set up as a macroscript so will need to be added as a button to your UI. The shader requires lightmap coords and texture coords in channels 1 and 2, and has a checkbox to state which is which.

Thanks in advance :).

Replies

  • TheFieryScythe
    So after getting someone with Maya and Turtle to bake me some example radiosity normal maps it turns out my maxscript is creating comparable lightmaps to turtles (normal basis wise). So the problems i'm seeing must be shader-side.

    The shader function I have for calculating the lightmap output is as follows:
    1. float3 CalculateDLM(float3 NormalMap,
    2. float3 DLightmap0,
    3. float3 DLightmap1,
    4. float3 DLightmap2)
    5. {
    6. float3 bumpBasis0 = float3( sqrt(2.0) / sqrt(3.0), 0.0, 1.0 / sqrt(3.0));
    7. float3 bumpBasis1 = float3( -1.0 / sqrt(6.0), 1.0 / sqrt(2.0), 1.0 / sqrt(3.0));
    8. float3 bumpBasis2 = float3( -1.0 / sqrt(6.0), -1.0 / sqrt(2.0), 1.0 / sqrt(3.0));
    9. float3 LightmapContribution = 0;
    10. LightmapContribution.x = saturate(dot(NormalMap, bumpBasis0));
    11. LightmapContribution.y = saturate(dot(NormalMap, bumpBasis1));
    12. LightmapContribution.z = saturate(dot(NormalMap, bumpBasis2));
    13. LightmapContribution *= LightmapContribution;
    14.  
    15. float3 diffuseLight = 0;
    16. float diffuseSum = dot(LightmapContribution, float3(1.0f, 1.0f, 1.0f));
    17. diffuseLight += DLightmap0 * LightmapContribution.x;
    18. diffuseLight += DLightmap1 * LightmapContribution.y;
    19. diffuseLight += DLightmap2 * LightmapContribution.z;
    20. diffuseLight /= diffuseSum;
    21.  
    22. return diffuseLight;
    23. }
    The NormalMap argument is unbiased before it is passed to the function.

    With that I get the following result in my test map:
    1.jpg

    ...which is pretty bad. If I flip the normals Y value it gets somewhat better with the walls rendering correctly:
    2.jpg

    ...but the floor is still wrong and I don't know why :(.
Sign In or Register to comment.