How to use Custom Plugin's, Scripts and Shaders.

Follow

Sometimes, users have more complex needs than what the conductor plugin can do by default.

  • Plugins
    • additional node scraping
    • compiled vs non-compiled
    • licensing
  • Pipeline scripts
    • environment variable
  • Custom Shaders

In order for these custom elements to be available for your job to be properly processed, they also must be uploaded as part of the job submission and available . 

Using custom scripts/plugins

Example 1: Using custom Python scripts in a render job

If you wanted a python script to be available to your render job, two things are needed:

  1. The python script needs to get uploaded to conductor
  2. An environment variable needs to be set in order to let the system know where to find the script on Conductor (in this case the PYTHONPATH).

Both of these things can be achieved by making changes to the config.yml file found in the conductor_client package.

Assuming that:

The python script lives here: /users/custom_scripts/script.py

The config.yml file looks something like

account: MyStudio

>>We need to add the python file the the file dependencies. In order to do this, we can add this to your config.yml file:

upload_paths:
- /users/custom_scripts/script.py

>>Now we need to add the environment variable to the config:

environment:
PYTHONPATH: /users/custom_scripts

>>Both the environment and upload_paths arguments are yaml lists, and can take multiple values. Let’s add an executable bash_file file and make it available via the PATH environment variable. This will give as a final config file of:

account: MyStudio

upload_paths:
- /users/custom_scripts/script.py
- /users/custom_scripts/bin/bash_file

environment:
PYTHONPATH: /users/custom_scripts
PATH: /users/custom_scripts/bin

Example 2: Dependency checking from custom node types

When an artist is submitting a job to conductor, the conductor plugin automatically sniffs out file dependencies that are required for rendering their file. However, most software packages (such as Maya) allow for custom plugins which potentially introduce dependency requirements that conductor may not be aware of.

Conductor searches for dependencies by looking for specific node types and reading from their attributes. These nodes types and their corresponding attributes are located in this file: conductor_client/conductor/resources/resources.yml
This resources file is a yaml file and must conform to yaml syntax. Some examples here.

Maya example

Here is an example of Maya dependency nodes that might be found in that file:

maya_dependency_attrs:
AlembicNode:
- abc_File
file:
- fileTextureName

From this example, you can see that AlembicNode and file node types are queried when seeking out dependencies. In order to have additional node types queried, simply add them to this list, including the attribute name(s) from which to read from.
For example, if you have node type called CustomNodeType and two attributes on it called input_file1 and input_file2 (where perhaps a user can input texture file paths) then add them to conductor’s maya dependency list like so:

maya_dependency_attrs:
AlembicNode:
- abc_File
file:
- fileTextureName
CustomNodeType:
- input_file1
- input_file2

Now when a job is submitted, any file paths that are found for the CustomNodeType will be uploaded and made available to the job.

Custom Maya Shaders

When using custom shaders like alShaders with Maya or Nuke, simply add their folder location and environment parameters to the Conductor Config yaml file. You can add shader files individually to limit the space requirement or push them all for availability.

Specifically, conductor needs to know about two things in order to make your custom shaders work.

  1. What files need to be uploaded (i.e. where the custom shaders are located on disk).
  2. What environment variable needs to be set in order for maya/arnold to find and load those files properly.

upload_paths:
    //YourStudio/pipeline/arnold/plugins/alshaders
  #your alshaders directory 

This will result in the inclusion of your custom shaders directory when submitting jobs, allowing your job to access these files during render time.

To specify individual custom shaders, stack upload paths to each individual file.

upload_paths:
   - //YourStudio/pipeline/arnold/plugins/alshaders/alflake.dylib
   - //YourStudio/pipeline/arnold/plugins/alshaders/alflake.mtd

For this example of installing alShaders, arnold/maya needs to become aware of these files (i.e. register/load your alshaders plugin), you'll also need to dictate some environment variables.  Per the alshaders docs,  it requires that you set the ARNOLD_PLUGIN_PATH and MTOA_TEMPLATES_PATH.

You can set environment variables for your Conductor jobs via the the config's environment key, e.g.
environment:
  ARNOLD_PLUGIN_PATH: //YourStudio/pipeline/arnold/plugins/alshaders/bin 
  MTOA_TEMPLATES_PATH: //YourStudio/pipeline/arnold/plugins/alshaders/ae


Once you have made these edits to your config, don't forget to restart maya before submitting to Conductor.

Nuke example

Similar to the Maya example above, Nuke has it’s own section in the resources file:

nuke_dependency_knobs:
Read:
- file
Group:
- vfield_file
- cdl_path

Add a new node type and it’s knobs to read from:

nuke_dependency_knobs:
Read:
- file
Group:
- vfield_file
- cdl_path
CustomNodeType:
- knob_name1
- knob_name2

Using an external Resource file

If you would rather maintain the resource outside of the conductor_client package, you can set the CONDUCTOR_RESOURCES_PATH to point to a different file path.
For example:
CONDUCTOR_RESORUCES_PATH=/users/custom_scripts/my_other_resource.yaml

This may be desirable when the conductor_client package is frequently updated and you don’t want to lose any custom changes that you made to the resources file. Note however, that this also means that you may miss out on any updates that were made to the original resources file (so it may be worth checking on it periodically).