Modding:Texture formatting

From bg3.wiki
(Redirected from Guide:Texture Formatting)
Jump to navigation Jump to search

This tutorial is from atmosfur

This guide has several sections related to the texture formats that BG3 uses, and how to export and prepare them in several programs. This guide is mainly about maps baked from custom 3D models and does not include skin textures currently.

The texture maps this guide covers are:


Additionally, to create a PM you will need:

  • AO (Ambient Occlusion Map)
  • Roughness Map
  • Metallic Map


This guide also includes a section on setting up maps by hand in GIMP if you do not have Substance Painter, or edited your textures by hand.

Substance Painter Export Preset[edit | edit source]

In case the BG3 export presets for Substance Painter do not work for you, or you have a pre-Adobe version of Substance Painter, here's how you can set up your own export presets in Substance Painter. I personally divided these into two separate presets due to my Basecolor and Mask map both pulling from different layers of basecolor information.

1. In the File -> Export Textures window, select the Configuration tab.
2. Click the small + icon next to "Presets" on the left side to create a new preset.
3. Create two R+G+B+A Output maps and one R+G+B Output map for the NM, PM and MSK map respectively.
4. On the right, you will see which Input maps you will need to drag and drop onto the Output maps R,G,B and A slots.
SPSettings1.webp
SPExport2.webp

5. Drag and drop the Input maps onto the corresponding slots in the Output maps tab:
Normal Map (Make sure to use the Converted maps - NormalDirectX):
R: Leave this empty
G: NormalDirectX, select Green channel
B: NormalDirectX, select Blue channel
A: NormalDirectX, select Red channel
SPExportNM.webp

Physical Map:
R: Metallic, select Gray channel
G: Roughness, select Gray channel
B: Ambient occlusion, select Gray channel
A: Leave this empty
SPPMexport.webp

Mask Map:
This is specifically for horn textures, you will need to check a vanilla mask of whatever you are texturing to see which channel has the mask for customizing color. In this case, horns only use the Green channel for that.

R: Leave this empty for horns
G: Base Color, select Green channel
B: Leave this empty for horns
6. As mentioned, for the Basecolor Map I create a new export preset. For this you just need one RGB+A Output map, and drag the Base Color Input map onto the RGB slot.
SPexportBM.webp

7. Go back to the "Export" tab of the window and select your newly created export presets from the drop down at the top, then just hit export. Double check your created textures in a program that lets you view channels (such as GIMP or Photoshop) to make sure they all got assigned correctly. You will still need to convert your textures to DDS now.

Creating the DDS files[edit | edit source]

Simple Conversion with Paint.NET[edit | edit source]

If your textures have all their channels set up correctly already, you just need to export them with the correct DDS compression in the editing program of your choice. I'm using Paint.NET as an example.

Basecolor Map Settings:
Choose BC1 (Linear,DTX1), make sure Generate Mip Maps is checked
BM DTX1.webp

Mask Map Settings:
Choose BC5 (Linear, Unsigned), make sure Generate Mip Maps is checked
MSKBC5.webp

Normal Map Settings:
Choose BC3 (Linear, DTX5), make sure Generate Mip Maps is checked
NMDTX.webp

Physical Map Settings:
Choose BC1 (Linear,DTX1), make sure Generate Mip Maps is checked
PMDTX.webp

Manually Assign Channels with GIMP[edit | edit source]

If you've created your textures in a program other than Substance Painter, you might need to manually assign the correct channels to your maps. Your best choice for doing this is probably GIMP, and I tried to make this part of the guide as beginner friendly as possible if you've never used the program before.
I recommend working on one map type at a time if you're following along with this guide. Again, this uses a horns texture as an example, the way other textures are set up in the game might be different, and it does not touch on the way skin textures are set up.

Normal Map[edit | edit source]

1. Open your Normal Map in GIMP, and click on the Channels tab to view the channels of your map. Depending on the program you created your normal map with, you might or might not have an Alpha channel already.
Nm1.webp

2. If you do not see an Alpha channel in your channels, switch back over to the Layers tab, right-click on your layer and select "Add Alpha Channel". This will add an empty Alpha channel.
Nm2.webp Nm3.webp

3. Back in your Channels tab, right-click on the "Red" channel and select "Channel to Selection".
Nm4.webp

4. Swap back to your Layer tab, then in your menu bar go to "Edit -> Clear" to delete the selected areas. This basically copies the contents of the Red channel to the Alpha channel.
Nm5.webp

5. Deselect your image by going to "Select -> None" in the menu bar.
Nm7.webp

6. Check your Channels tab, your Alpha channel should now look like your Red channel.
Nm8.webp

7. In the menu bar, go to "Colors -> Components -> Decompose". This is going to create and open a new version of your file, with all the channels separated into individual layers.
Nm9.webp

8. In the pop-up window select RGBA as the Color mode, then hit OK.
Nm10.webp

9. You should now see a file that looks like your normal map, but in greyscale. In the layers tab you will see that each channel has its own layer now.
Nm11.webp

10. Before the next step, make sure that your Foreground color is set to black. If it is not, you can click the small icon of a black and white square underneath the color selector to reset it.
Nm12.webp

11. Click on the "red" layer to set it to your active layer, then in the menu bar go to "Edit -> Fill with FG Color" to fill the layer with black.
Nm13.webp Nm14.webp

12. Go to "Colors -> Components -> Recompose" in the menu bar to recompose the layers back into channels in your original file.
Nm17.webp

13. Go back to the original file and check the Channels tab to make sure the red channel is just black (empty) now.
Nm18.webp

14. If your original file seems to still be the same as before, make sure that in the channels tab none of the channels are selected/highlighted. If any of them are, just click on them again to de-select them, then go back to the Decomposed Layers file and repeat step 12 to recompose it.
Nm15.webp Nm16.webp

15. Your recomposed file is now ready to export. Go to "File -> Export As..."
Nm19.webp

16. Select a save location for your file, and name it to whatever you choose. Check the vanilla game textures to see what naming conventions are used for the files the textures are for. I personally also export a .png version to keep as a backup, but you will need a .dds file for the game. You can either rename your file with the .dds extension, or click "Select File Type (By Extension)" at the bottom to select the .dds file type. Once ready, just click "Export" to save your texture. A window will pop-up with export options. For .png, just leave everything at default.
Nm20.webp
Nm21.webp

17. To correctly export your Normal map as a .dds file, make sure to select "BC3/DTX5" for Compression, and "Generate mipmaps" for Mipmaps. Set the Filter under Mipmap Options to "Kaiser".
Nm22.webp

Physical Map[edit | edit source]

To create the Physical Map, you will need to have these three maps:

  • AO Map
  • Roughness Map
  • Metallic Map

1. Open all three maps in GIMP. (If your Metallic Map is just black just skip opening it)
Pm23.webp

2. Go to your Roughness Map and in the menu bar, go to "Image -> Mode -> RGB" to convert your image from Grayscale mode to RGB mode. This creates the RGB channels needed.
Pm25.webp

3. Right-click on your layer and select "Add Alpha Channel". This will add a blank Alpha channel.
Pm26.webp Pm27.webp

4. In the menu bar, go to "Colors -> Components -> Decompose". This is going to create and open a new version of your file, with all the channels separated into individual layers.
Pm28.webp

5. In the window that pops up, select "RGBA" as the Color mode, then hit OK.
Pm29.webp

6. You should now see a second file that looks like your roughness map, with each channel as a separate layer.
Pm30.webp

7. If your Metalness map was just black, you can simply fill the "red" layer with black. If your Metalness map has information in it, follow the steps for the AO map instead to insert the Metalness map into the "red" layer.
To fill the layer with black, click on the "red" layer to set it to active, then go to "Edit -> Fill with FG Color" in the menu bar, making sure that your foreground color is set to black.
Pm31.webp Pm32.webp

8. Now it's time to insert the AO map. To make things easier, hide both the "red" and "green" layer by clicking the little eye shaped icon next to them.
Pm33.webp

9. Now swap over to your AO map file and in the menu bar go to "Select -> Select All" to select the entire image.
Pm34.webp Pm35.webp

10. In the menu bar, go to "Edit -> Copy" (or hit Ctrl+C) to copy your AO map image.
Pm36.webp

11. Swap back over the Decomposed Layers file and click on the "blue" layer to set it to the active layer.
Pm37.webp

12. In the menu bar, go to "Edit -> Paste In Place" to paste your copied AO map.
Pm38.webp

13. You should see your AO map pasted in the file now as a "Floating Selection".
Pm39.webp

14. In the layer tab, click the small anchor icon at the bottom to anchor the pasted layer onto the one below it.
Pm40.webp Pm41.webp

15. Your "blue" layer should now have merged with the pasted image. Merging the layers together this way ensures that they will Recompose properly in the end. If your Metalness map has information in it, paste it into the "red" layer the same way as the AO map.
Pm42.webp

16. Click on the small eye icons next to the layers you hid before to make them visible again.
Pm43.webp

17. Go to "Colors -> Components -> Recompose" in the menu bar to recompose the layers back into channels in your original file.
Nm17.webp

18. Go back to the original file and check the Channels tab to make sure it recomposed properly - your PM map should now also look something like this.
Pm45.webp

19. If your original file seems to still be the same as before, make sure that in the channels tab none of the channels are selected/highlighted. If any of them are, just click on them again to de-select them, then go back to the Decomposed Layers file and repeat step 17 to recompose it.
Nm15.webp Nm16.webp

20. Your channel order for the finished PM should be Metalness in Red, Roughness in Green, AO in Blue, and nothing in Alpha.
Pm46.webp

21. Your recomposed file is now ready to export. Go to "File -> Export As..."
Nm19.webp

22. Select a save location for your file, and name it to whatever you choose. Check the vanilla game textures to see what naming conventions are used for the files the textures are for. I personally also export a .png version to keep as a backup, but you will need a .dds file for the game. You can either rename your file with the .dds extension, or click "Select File Type (By Extension)" at the bottom to select the .dds file type. Once ready, just click "Export" to save your texture. A window will pop-up with export options. For .png, just leave everything at default.
Nm20.webp
Nm21.webp

23. To correctly export your Physical map as a .dds file, make sure to select "BC1/DTX1" for Compression, and "Generate mipmaps" for Mipmaps. Set the Filter under Mipmap Options to "Kaiser".
Pm47.webp

Basecolor Map[edit | edit source]

1. Open your Basecolor Map in GIMP. Right-click on the layer and select "Add Alpha Channel". This will add an empty Alpha channel.
Bm49.webp Bm50.webp

2. Your file is now ready to export. Go to "File -> Export As..."
Nm19.webp

3. Select a save location for your file, and name it to whatever you choose. Check the vanilla game textures to see what naming conventions are used for the files the textures are for. I personally also export a .png version to keep as a backup, but you will need a .dds file for the game. You can either rename your file with the .dds extension, or click "Select File Type (By Extension)" at the bottom to select the .dds file type. Once ready, just click "Export" to save your texture. A window will pop-up with export options. For .png, just leave everything at default.
Nm20.webp
Nm21.webp

4. To correctly export your Basecolor map as a .dds file, make sure to select "BC1/DTX1" for Compression, and "Generate mipmaps" for Mipmaps. Set the Filter under Mipmap Options to "Kaiser".
Bm51.webp

Mask Map[edit | edit source]

The channels that mask maps use in the game vary depending on what the mask is used for. This example is just for horn textures, so check the vanilla game textures first to find out what channels you need. In any case, mask maps do not need an Alpha channel, so all you have to do for them is export as a .dds file.
Either way, it doesn't hurt to check the Channels tab to make sure you've only got information in the channels that are supposed to have it, i.e. just the "green" channel for horn mask maps.
Msk53.webp

1. Your file is now ready to export. Go to "File -> Export As..."
Nm19.webp

2. Select a save location for your file, and name it to whatever you choose. Check the vanilla game textures to see what naming conventions are used for the files the textures are for. I personally also export a .png version to keep as a backup, but you will need a .dds file for the game. You can either rename your file with the .dds extension, or click "Select File Type (By Extension)" at the bottom to select the .dds file type. Once ready, just click "Export" to save your texture. A window will pop-up with export options. For .png, just leave everything at default.
Nm20.webp
Nm21.webp

3. To correctly export your Mask map as a .dds file, make sure to select "BC5/ATI2 (3Dc)" for Compression, and "Generate mipmaps" for Mipmaps. Set the Filter under Mipmap Options to "Kaiser".
Msk54.webp

Export Settings TL;DR[edit | edit source]

If you already know how to do all the channel editing and just need a quick reference on how these maps are set up and exported.

Normal Map Channels:

  • Red - Nothing
  • Green - Green
  • Blue - Blue
  • Alpha - Red

Physical Map Channels:

  • Red - Metalness
  • Green - Roughness
  • Blue - AO
  • Alpha - Nothing

Basecolor Map Channels:

  • Red - Red
  • Green - Green
  • Blue - Blue
  • Alpha - Nothing

Mask Map Channels:

  • Check vanilla textures for whatever you're masking. These usually don't need an Alpha channel.

DDS Export Settings:

  • Normal Map:
  • BC3 DTX5

  • Physical Map:
  • BC1 DTX1

  • Basecolor Map:
  • BC1 DTX1


  • Mask Map:
  • These again depend on the type of mask you're creating. I'll be collecting as much info for different mask maps here as I find.
  • BC5 Unsigned (Paint.NET) / BC5 ATI2 (GIMP) for Horn masks
  • BC7 RGBA 8bpp | Explicit alpha (Photoshop) / BC7 sRGB or BC7 Linear (Paint.NET) for Shared Scars MSK (thanks to aurebuen for the info)

Icon/Atlas Export Settings[edit | edit source]

If using Paint.net and your icon atlas ends up invisible in game when using DXT1/DXT5 try using the below:

  • First make sure your uuids are unique and are not clashing with another mod
  • if that doesn't work use this format

Atlas Paintnet Export Settings.webp