Isotropix Forums

How to find all the external dependencies in a context

Clarisse Scripting related topics

How to find all the external dependencies in a context

Unread postby bvz2000 » Thu Sep 08, 2016 2:15 am

I am trying to get a list of all the external dependencies in a context (along with their nodes).

I guess that I am just going to have to do something along the lines of:

Get all nodes
Check each node to see whether it is an instance of another node that is external to the root context I am checking.
Check each node to see if it has an input somewhere that is another node that is external to the root context I am checking.


Is there a way to query a node and get a list of all the nodes that feed into it (either directly or recursively)?

For example, the obvious ones are Combiners, Group nodes, Scatterers, etc. But most of the Material Editor nodes can have inputs. Even Cameras can have parent nodes as well as constraints. How can I effectively figure out all the dependencies that lead into a single node?

Thanks.

Edit:
To clarify, what I am trying to do is copy (not move) all of the dependent nodes into the context I am checking, and then re-link the nodes that have external dependencies to point to these copied nodes.
bvz2000
 
Posts: 340
Joined: Thu Nov 13, 2014 7:05 pm

Re: How to find all the external dependencies in a context

Unread postby bvz2000 » Tue Sep 13, 2016 2:09 am

Ok, I mostly have this working. The last little bit that seems to be giving me trouble is that I cannot seem to figure out how to get back all of the object references from something like a combiner.


When I try to extract the objects attribute from the combiner I use:


Code: Select all
combinerNode.get_attribute("objects").get_object()



But that only returns the first item in the combiner. What I want is a list of all the objects in this attribute.

More importantly, I need to know when I query an attribute whether that attribute will give me back a single item (like parent) or multiple items (like objects). Is there a single call I can make on both of those parameters that will always give me a list - only a single item in the list for parent and multiple items in the list for objects?
bvz2000
 
Posts: 340
Joined: Thu Nov 13, 2014 7:05 pm

Re: How to find all the external dependencies in a context

Unread postby caudru » Tue Sep 13, 2016 10:01 am

Hi,

you should give a try to OfContext::get_external_dependencies(...) to access external items of a context as you expect.

Code: Select all
ctx = ix.get_item("project://groups")

ext_refs = ix.api.OfItemSet()
ext_sources = ix.api.OfItemSet()
ctx.get_external_dependencies(ext_refs, ext_sources)

if (ext_refs.get_count() > 0):
    print "external dependencies : {0}".format(ext_refs.get_count())
    for i in range(0, ext_refs.get_count()):
        print "- {0}".format(ext_refs[i].get_full_name())

if (ext_sources.get_count() > 0):
    print "external sources : {0}".format(ext_sources.get_count())
    for i in range(0, ext_sources.get_count()):
        print "- {0}".format(ext_sources[i].get_full_name())


you can also try:
OfObjectFactory::get_items_dependencies(...)
OfObjectFactory::get_items_inputs(...)
OfObjectFactory::get_items_outputs(...)

Code: Select all
obj = ix.get_item("project://groups/Asset_groups/character_F35Hawk_body")

obj_array = ix.api.OfItemArray(1)
obj_array[0] = obj
#

item_dependencies = ix.api.OfItemVector()
ix.application.get_factory().get_items_dependencies(obj_array, item_dependencies, False)

if (item_dependencies.get_count() > 0):
    print "item dependencies : {0}".format(item_dependencies.get_count())
    for i in range(0, item_dependencies.get_count()):
        print "- {0}".format(item_dependencies[i].get_full_name())

item_inputs = ix.api.OfItemVector()
ix.application.get_factory().get_items_inputs(obj_array, item_inputs, False)

if (item_inputs.get_count() > 0):
    print "item inputs : {0}".format(item_inputs.get_count())
    for i in range(0, item_inputs.get_count()):
        print "- {0}".format(item_inputs[i].get_full_name())

item_outputs = ix.api.OfItemVector()
ix.application.get_factory().get_items_outputs(obj_array, item_outputs, False)

if (item_outputs.get_count() > 0):
    print "item outputs : {0}".format(item_outputs.get_count())
    for i in range(0, item_outputs.get_count()):
        print "- {0}".format(item_outputs[i].get_full_name())


and to get all the values of an attribute, you can use the following:
Code: Select all
item = ix.get_item("project://groups/Asset_groups/F35_ace_geo_GRP")
attr = item.get_attribute("references")

also_get_disabled_items = True

value_count = attr.get_value_count(not also_get_disabled_items)
print "attr '{0}' value count : {1}".format(attr.get_full_name(), value_count)
for i in range(0, value_count):
    print attr.get_object(i, ix.api.OfAttr.VALUE_PAGE_CURRENT, not also_get_disabled_items)


values_array = ix.api.OfObjectArray()
attr.get_values(values_array, ix.api.OfAttr.VALUE_PAGE_CURRENT, not also_get_disabled_items)

value_count = values_array.get_count()
print "attr '{0}' value count : {1}".format(attr.get_full_name(), value_count)
for i in range(0, value_count):
    print values_array[i]


Regards
User avatar
caudru
 
Posts: 70
Joined: Thu Feb 05, 2015 10:27 am

Re: How to find all the external dependencies in a context

Unread postby bvz2000 » Tue Sep 13, 2016 7:03 pm

Wow. That is super helpful! Thank you.

I am going to dig through this today.
bvz2000
 
Posts: 340
Joined: Thu Nov 13, 2014 7:05 pm

Re: How to find all the external dependencies in a context

Unread postby bvz2000 » Sat Sep 02, 2017 9:46 pm

Ok, I am revisiting this process after a long while.

Previously I had mostly gotten it to work through a lot of code and by the time you had offered your quicker way of doing it I had already finished most of my stuff.

Now I am rewriting much of my old code and I would like to use your method.

The problem I am trying to solve is this:

I want to copy (not move) any external dependencies of the nodes inside a context into that context and then re-link these original nodes (that originally pointed to the external dependencies) to now point to these newly copied (and no longer external) dependencies.

Your first bit of code successfully gives me a list of all the external dependencies. But as far as I can tell, this is just a list of these nodes with no way of finding out which attribute of which original node inside the context was referencing them.

So that means I have no quick way of re-linking the these attributes to a different destination (for example, copies of these external dependencies that I would make).

So what I am planning is:

1) Get a list of external dependencies using "get_external_dependencies"
2) Step through this list and get each of these nodes' outputs using "get_items_outputs"
3) Check each of these outputs to see if it lives in the original context I am testing by comparing URL's
4) If the output node does live inside the original context (i.e. it is one of the nodes inside the original context that had an external dependency), get the attributes from this node, looking for the original external dependency node. This will give me the attribute object that points to the external dependency
5) Copy the external dependency node into the original context.
6) Relink the attribute object to point to this copied node.
7) Move on to the next output for this external dependency and do it all again (it may be linked to more than once within the original context).
8) Move on to the next external dependency and do it all again.
9) Somehow make sure that if I have multiple nodes inside the original context that point to the same external dependency that I don't duplicate this external dependency inside the original context more than once.


This seems fairly complex (and not all that dissimilar to the method I am already using). Is there a quicker, cleaner way of accomplishing my goal?

(My previous method was to simply list every node in the context, check any properties that might reference another node, if that property referenced a node, and that node's URL was not inside of the current context, copy that node and relink the property. Sounds simple, but it was a LOT of code to make it work reliably).

Thanks!
bvz2000
 
Posts: 340
Joined: Thu Nov 13, 2014 7:05 pm

Re: How to find all the external dependencies in a context

Unread postby bvz2000 » Sun Sep 03, 2017 1:29 am

Just fyi:

This is the code that I am currently using. It works as far as I have been able to test till now, but I'm wondering whether I am doing it the hard way and whether there is a better and easier way using some of the code above.

Code: Select all
# --------------------------------------------------------------------------------------------------
def copyNode(sourceObj, destContextObj):
    """
    Copies a node object (as given by sourceObj) to a context (as given by destContextObj).

    :param sourceObj:
    :param destContextObj:
    :return:
    """

    # create an instance of the source object in the destination context
    instance = destContextObj.add_instance(sourceObj)

    # transform the new instance into a copy
    instance.make_local()

    # Return this new object
    return instance


# --------------------------------------------------------------------------------------------------
def getAllAttrObjects(sel, *typeFiltersL):
    """
    Returns a python list of all the attribute objects for a given selection. If typeFilters are
    given (as integers), then the list will only include attributes of those types.

    :param sel:
    :param typeFiltersL:
    :return:
    """

    attrsL = list()
    attrCount = sel.get_attribute_count()
    for i in range(0, attrCount):
        attr = sel.get_attribute(i)

        if len(typeFiltersL) != 0:
            if attr.get_type() in typeFiltersL:
                attrsL.append(attr)
                continue
        else:
            attrsL.append(attr)

    return attrsL



# --------------------------------------------------------------------------------------------------
def getAllAttrTextureConnections(attr, obj=None, pythonList=True):
    """
    Returns a python list OR Clarisse array of all the connected textures to a particular attribute
    object. The attribute may either be passed as an attribute object, or as an attribute name. If
    passed as a name, the obj that contains this attribute must also  be passed. If pythonList is
    True, then it will return a python list of attribute values. If false, it will return a built
    in Clarisse array.

    :param attr:
    :param pythonList:
    :return:
    """

    textureConnectionsL = list()

    # If they passed an attribute name, convert that to an attribute object
    if isinstance(attr, str):
        attr = obj.get_attribute(attr)

    # Get all the texture connections
    textureConnection = attr.get_texture()
    if textureConnection is not None:
        textureConnectionsL.append(textureConnection)

    if pythonList:
        textureConnectionsL

    clarisseArray = ix.api.OfItemArray(len(textureConnectionsL))
    for i in range(0, len(textureConnectionsL)):
        clarisseArray[i] = textureConnectionsL[i]
    return clarisseArray


# --------------------------------------------------------------------------------------------------
def getAllAttrValues(attr, obj=None, pythonList=True):
    """
    Returns a python list OR Clarisse array of all the values for the specific attribute. The
    attribute may either be passed as an attribute object, or as an attribute name. If passed as a
    name, the obj that contains this attribute must also  be passed. If pythonList is True, then it
    will return a python list of attribute values. If false, it will return a built in Clarisse
    array.

    :param attr:
    :param obj:
    :param pythonList:
    :return:
    """

    valuesL = list()

    # If they passed an attribute name, convert that to an attribute object
    if isinstance(attr, str):
        attr = obj.get_attribute(attr)

    # Get all the values of this attribute
    valuesArray = ix.api.OfObjectArray()
    attr.get_values(valuesArray)

    if pythonList:

        # Convert this to a python list
        for i in range(0, valuesArray.get_count()):
            valuesL.append(valuesArray[i])

        return valuesL

    return valuesArray


# --------------------------------------------------------------------------------------------------
def getExternalLinks(refContext, nodesL):
    """
    Returns a dictionary of all the external dependencies in a given list of nodes, where the key is
    the attribute object of the project item that has the external reference, and the value is a
    clarisse array of objects, at least one of which will be an external dependency (there may some
    that are NOT external in this list as well). This method will only search those nodes that are
    passed to in via the nodesL list, and will consider any linked nodes that are not in the context
    passed in 'refContext' as being external.

    :param refContext: the reference context that will be used to consider whether something is
                       "external" or not.
    :param nodesL: a list of nodes that will be examined to see if they contain external links.
    :return:
    """

    extLinksD = dict()

    for node in nodesL:

        # Get a list of all the attributes for this node
        attrsL = getAllAttrObjects(node, ix.api.OfAttr.TYPE_OBJECT,
                                   ix.api.OfAttr.TYPE_REFERENCE)

        # Check each attr to see if it has an external dependency
        for attr in attrsL:

            # Get a list of all the referenced objects in this attribute, external or not.
            refObjsArray = getAllAttrValues(attr, pythonList=False)

            # If any of them are external to the current context, store the entire array
            for i in range(0, refObjsArray.get_count()):
                refObj = refObjsArray[i]
                if refObj is None:
                        continue
                if (not refObj.is_within(refContext, True) and
                        not refObj.get_full_name().startswith("project://default/")):
                    extLinksD[attr] = refObjsArray  # Store entire array
                    break  # No need to go an further because we stored the whole array already

    return extLinksD



# --------------------------------------------------------------------------------------------------
def getExternalTextures(refContext, nodesL):
    """
    Returns a dictionary of all the material nodes in nodesL that point to textures external to the
    passed context. The key is the attribute object that points to an externally dependent texture,
    and the value is the texture object that lives external to the context being checked. This
    method will only search those nodes that are passed to in via the nodesL list, and will consider
    any linked texture nodes that are not in the context passed in 'refContext' as being external.

    :param refContext:
    :param nodesL:
    :return:
    """

    extTexturesD = dict()

    for node in nodesL:

        # Get a list of all the attribute objects for this node of any type
        attrsL = getAllAttrObjects(node)

        # Check each attr to see if it has an external texture dependency
        for attr in attrsL:

            # Get a list of all the texture objects, external or not.
            txtObjsL = getAllAttrTextureConnections(attr, pythonList=False)

            for txtObj in txtObjsL:
                if txtObj is None:
                    continue
                if not txtObj.is_within(refContext, True):
                    extTexturesD[attr] = txtObjsL  # Store ALL objects in this attr, external or not
                    break  # No need to go an further because we stored the whole array already

    return extTexturesD



# --------------------------------------------------------------------------------------------------
def getExternalSources(refContext, nodesL):
    """
    Returns a dictionary of all the external instances in a given list of nodes, where the key is
    the node that is an instance, and the value is a clarisse array of objects, at least one of
    which will be the source of this instance. This method will only search those nodes that are
    passed to in via the nodesL list, and will consider any source nodes that are not in the context
    passed in 'refContext' as being external.

    :param refContext:
    :param nodesL:
    :return:
    """

    extSourcesD = dict()

    for node in nodesL:

        # Check to see if the object is an instance
        if node.is_instance():

            # Check to see if it is an external instance
            if not node.get_source().is_within(refContext, True):
                # Add it to the list
                extSourcesD[node] = node.get_source()

    return extSourcesD


# --------------------------------------------------------------------------------------------------
def retargetExternalLinks(refContext, extLinksD, destContext, texture=False, completedD=dict()):
    """
    Does the actual localization for the selected external links. refContext is the parent context
    that we are considering to be the "base" against which we are localizing. extLinksD is a
    dictionary where the key is the attribute object and the value is a Clarisse array of nodes that
    the attribute links to. destContext is the context into which we will be copying the external
    nodes. completedD is a dictionary where the key is a complete Clarisse URL to a specific node
    that has already been processed, and the value is the newly copied item that now lives in the
    destContext context. Returns a dictionary of all the nodes that were copied into the destContext
    context.

    :param refContext:
    :param extLinksD:
    :param destContext:
    :param texture: If this is a texture link or not. Defaults to False.
    :param completedD:
    :return:
    """

    addedD = dict() # key = original external path, value = local path where node was copied to

    # Step through each attribute that contains an externally referenced node in it somewhere
    for attr in extLinksD.keys():

        # Get the array of objects that this attribute contains
        extObjsArray = extLinksD[attr]

        # Process each of the objects in this array
        for i in range(0, extObjsArray.get_count()):

            extObj = extObjsArray[i]

            # Extract the path
            extObjPath = extObj.get_full_name()

            # Check if this path really is is external (it is possible to get back a list that
            # contains a mix of external and non-external nodes)
            if (not extObj.is_within(refContext, True) and
                    not extObjPath.startswith("project://default/")):

                # Check to see if this external object has already been processed (copied locally)
                if extObjPath not in completedD.keys() and extObjPath not in addedD.keys():

                    # It hasn't, so copy it locally
                    newTargetNode = copyNode(extObj, destContext)

                    # Remember this node
                    addedD[extObjPath] = newTargetNode

                else:

                    # It has already been copied locally, so extract the previously copied node
                    if extObjPath in completedD.keys():
                        newTargetNode = completedD[extObjPath]
                    else:
                        newTargetNode = addedD[extObjPath]

            else:

                # This is NOT an externally referenced node, so keep it un-altered
                newTargetNode = extObj

            # Now retarget the original attribute to point to this new node
            if texture:
                attr.set_texture(newTargetNode)
            else:
                attr.set_object(newTargetNode, i)

            # Drop a breadcrumb
            # setCustomAttribute(newTargetNode, "CLAM", "OriginalClarissePathOfObject",
            #                    extObj.get_full_name())

    return addedD


# --------------------------------------------------------------------------------------------------
def retargetExternalSources(context, extSourcesD, destContext, completedD=dict()):
    """
    Does the actual localization for the selected external sources (where "source" refers to the
    source of an instanced node). refContext is the parent refContext that we are considering to be
    the "base" against which we are localizing. extSourcesD is a dictionary where the key is the
    instanced object and the value is the source object that the instance points to.
    destContext is the context into which we will be copying the external nodes. completedD is a
    dictionary where the key is a complete Clarisse URL to a specific node that has already been
    processed, and the value is the newly copied item that now lives in the destContext context.
    Returns the addedD dictionary of all the nodes that were copied into the destContext context.

    :param refContext:
    :param extSourcesD:
    :param destContext:
    :param completedD:
    :return:
    """

    addedD = dict()

    # Step through each instance that has an external source and copy it.
    for instance in extSourcesD.keys():

        # Check to see if this external object has already been processed (copied locally)
        extObjPath = extSourcesD[instance].get_full_name()
        if extObjPath not in completedD.keys() and extObjPath not in addedD.keys():

            # It hasn't, so copy it locally
            newTargetNode = copyNode(instance, destContext)

            # Remember this node
            addedD[extObjPath] = newTargetNode

        else:

            # It has already been copied locally, so extract the previously copied node
            if extObjPath in completedD.keys():
                newTargetNode = completedD[extObjPath]
            else:
                newTargetNode = addedD[extObjPath]

        # Now retarget the original instance to point to this new node
        ix.cmds.MakeInstance([instance.get_full_name()], newTargetNode.get_full_name())

        # Drop a breadcrumb
        # setCustomAttribute(newTargetNode, "CLAM", "OriginalInstanceSource", extObjPath)

    return addedD






# --------------------------------------------------------------------------------------------------
def localizeExternalDependencies(refContext, searchContext=None, completedD=dict()):
    """
    If a context contains items that depend on project items that are outside of its scope (are
    external to it), then this will copy those items to a sub-context in the original context called
    "_externally_referenced_nodes", and re-link the original project items to point to these new,
    copied items.

    The function will repeat as many times as needed until no more external dependencies are found.
    (i.e. if one of the newly copied nodes also depends on an external node, that will be caught
    during the next pass - a process that will continue until no further external dependencies are
    found). The dict: completedD is a dictionary whose key is the original path of an object that
    was copied, and whose value is the new, copied object. This dict must be passed to recursive
    runs of this function to prevent any external nodes that are referenced more than once becoming
    multiple nodes when copied.)

    :param refContext: the reference context that will be used to consider whether something is
                       "external" or not.
    :param serachContext: the actual context that will be searched to see if it has external
                          dependencies. This may be different than the refContext when running
                          recursively. Typically it will be the same as refContext when called from
                          outside of this function. If nothing is passed to this parameter (or
                          it is passed a None) it will automatically use the refContext as the
                          search context.
    :param completedD: a dictionary (key = original URL, value = copied URL) of any external
                       dependencies that have already been copied during previous recursions.

    :return:
    """

    # If they did not suppy a searchContext (or they gave None as a searchContext) then use the
    # refContext as the searchContext
    if searchContext is None:
        searchContext = refContext

    # Get a list of all the nodes in the search context
    nodesL = list()
    nodesArray = ix.api.OfObjectArray()
    searchContext.get_all_objects("ProjectItem", nodesArray)
    for i in range(nodesArray.get_count()):
        nodesL.append(nodesArray[i])


    addedD = dict()

    # Get all the external dependencies stored as links in the reference context
    extLinksD = getExternalLinks(refContext, nodesL)

    # Also get external texture nodes separately from the above process
    extTexturesD = getExternalTextures(refContext, nodesL)

    # Also get all of the instances that point to nodes external to the reference context
    extSourcesD = getExternalSources(refContext, nodesL)

    # If there are no external dependencies, bail (this will also stop recursion on this branch)
    if len(extLinksD) == 0 and len(extSourcesD) == 0 and len(extTexturesD) == 0:
        return None, None

    # Create a new context at the top level of the context being tested called
    # "_externally_referenced_nodes"
    extRefShortName = "_externally_referenced_nodes"
    extRefNodesContextName = refContext.get_full_name() + "/" + extRefShortName
    extRefNodesContext = refContext.item_exists(extRefShortName)
    if extRefNodesContext == None:
        extRefNodesContext = ix.create_context(extRefNodesContextName)

    # Retarget all of the external items (and store the copied nodes in addedD)
    addedD.update(retargetExternalLinks(refContext, extLinksD, extRefNodesContext, completedD))
    addedD.update(retargetExternalLinks(refContext, extTexturesD, extRefNodesContext, True,
                                        completedD))
    addedD.update(retargetExternalSources(refContext, extSourcesD, extRefNodesContext, completedD))

    # Update completedD with these newly added nodes
    completedD.update(addedD)

    # Now re-run this same command again to look for additional external dependencies
    # We only have to check the extRefNodesContext reference since that is the only place new
    # nodes would have appeared since the last time we ran.
    # We don't care about what the function returns, since everything of value is stored in
    # completedD
    localizeExternalDependencies(refContext, extRefNodesContext, completedD)

    # Now we have a newly created context, plus some newly created nodes. These need to be
    # communicated back to whoever called this function in case that function needs to know what
    # changed.
    return completedD.values(), extRefNodesContext




sel = ix.selection[0]
if sel.is_context():
    localizeExternalDependencies(sel)
else:
    print sel.get_name() + " is not a context."
bvz2000
 
Posts: 340
Joined: Thu Nov 13, 2014 7:05 pm


Return to Scripting