Location Setup

[eight]

Hi, I've read the locations tutorial for the new API, but I still find it hard to understand. My goal was to simply show the correct path for a component in the web interface based on the platform the user is on. So the component is published with a relative path and based on whether the user is looking at the ftrack web interface from Windows, Mac, or Linux, the prefix would change to use the correct mount point for them.

 

So something like:

if platform == 'Windows':    
	prefix = r'\\SHARED\jobs'
elif platform == 'Mac':    
	prefix = '/Volumes/jobs'
elif platform == 'Linux':    
	prefix = '/mnt/jobs'

I can't figure out how to do this.

[Mattias Lagergren]

Hi, have you built your own location or are you using one of the locations shipped with ftrack?

[eight]

I created my own location. The code is similar to the tutorial, but there is no resolver example for the new API, so here's my code for that:

def mainCallback(event):    
	query = 'Location where name is "{name}"'.format(name=event['data']['locationName'])    
	location = session.query(query).one()    
	component = session.get('Component', event['data']['componentId'])    
	resolvedPath = location.get_filesystem_path(component)        
	return {'path': resolvedPath}

session.event_hub.subscribe('topic=ftrack.location.request-resolve', mainCallback)

 

[Mattias Lagergren]

In event['data']['platform'] you will have the Windows or Linux - where Linux is both Mac and Linux.

 

An alternative could be to run it in Connect under each individual users Connect:

session.event_hub.subscribe('topic=ftrack.location.request-resolve and source.user.id="<user id of connect user>"', mainCallback)

 

With this solution you can use the user's python platform to figure out the user platform to resolve to.

[eight]

So, I would instead publish components using a relative path like:

assetVersion.create_component(relativePath, {'name': componentName}, location='my.location')

Then, for the path resolver I would do this?

def mainCallback(event):    
	query = 'Location where name is "{name}"'.format(name=event['data']['locationName'])    
	location = session.query(query).one()    
	component = session.get('Component', event['data']['componentId'])   
	relativePath = location.get_filesystem_path(component)
	
	if event['data']['platform'] == 'Windows':
		resolvedPath = os.path.join(r'\\SHARED\jobs', relativePath)
		resolvedPath = resolvedPath.replace('/', '\\')
	elif event['data']['platform'] == 'Linux':
		resolvedPath = os.path.join('/mnt/jobs', relativePath)
		resolvedPath = resolvedPath.replace('\\', '/')
	
	return {'path': resolvedPath}

If this is the case, then I'm confused what the purpose of prefix is for the constructors of Accessor and Structure when configuring locations.

Also, there's no way to tell if the platform is a Mac? I thought Macs used /Volumes instead of /mnt?

[Mattias Lagergren]

The platform dependent disk accessor prefix makes more sense when you are using locations to publish/get data on a certain platform (e.g. in publishing tools or in Maya). It is not as intuitive when you are using it in the resolver case on different platforms. However, try and change the accessor prefix before calling get_filesystem_path:

# If linux:location.accessor.prefix = '/mnt/jobs'

 

 

Also, there's no way to tell if the platform is a Mac? I thought Macs used /Volumes instead of /mnt?

 

At the moment Mac and Linux is grouped under "Linux", but we have a feature that we are currently working on that may change this to the better.

[eight]

Okay. Thank you for helping me understand how this all works!

[eight]

I could use more help since I'm now trying to transfer components between two locations.

I'm using this example: https://www.ftrack.com/portfolio/transfer-components . However, the custom locations are not showing up. I know it's because of ftrack.getLocations(excludeInaccessible=True) . If I change it to False, my locations will show up. But why are my locations considered inaccessible?

I'm creating locations like this:

query = 'Location where name is "{name}"'.format(name=locationName)
location = session.query(query).one()
location.accessor = DiskAccessor('')
location.structure = LocationStructure(prefix='')
location.priority = 50

Shouldn't the DiskAccessor be enough for the Location to be considered accessible?

[Mattias Lagergren]

Not sure exactly how your setup looks like at this point, but on top of my head:

• ftrack.getLocations sounds like the legacy api, while your location creation code is in our new ftrack-python-api. They do not seamlessly work together, instead you will have to make the location available as a plugin in the legacy api as well.
• In the legacy api you will have to call ftrack.setup() before running ftrack.getLocations. ftrack.setup() will look in the FTRACK_LOCATION_PLUGIN_PATH for your location plugins.

Please let me know how it goes

[eight]

I changed everything to the legacy API and I'm able to get the custom locations to show up, but transfers do not work.

I get: 

Quote

AccessorResourceNotFoundError: Resource not found: path/to/file/filename.nk

Do I need to create my own accessor to transfer files between two different studios? Or is there something that magically makes it work like piping the data through the ftrack server somehow?

[Mattias Lagergren]

1 hour ago, eight said:

Do I need to create my own accessor to transfer files between two different studios? Or is there something that magically makes it work like piping the data through the ftrack server somehow?

The location system can be used to represent the both storages, but no magic happens when it comes to transferring across multiple sites.

For example if you have Studio A and Studio B with their corresponding location A and B.

If you want to transfer data from location A to B from a script running at Studio A, location B's storage must be accessible with the accessor. If you're using a plain disk accessor this requires you to have mounted the storage of location B. If you cannot mount the storage of location B you will need to find other means of transfer, FTP, rsync or something else. This means that your location plugins will look differently depending on which studio you're at.

Studio A

Location A: disk accessor with mount point that is accessible

Location B: ftp/rsync/other accessor

Studio B

Location A: ftp/rsync/other accessor

Location B: disk accessor with mount point that is accessible

[eight]

Thank you! This makes a lot more sense now.

Is it possible to port the example transfer code to the new API or are there some things in there that the new API doesn't support? I can figure out how to convert most of it, but some things that stand out at me are:

ftrack.getLocations(excludeInaccessible=True)

and 

targetLocation.addComponent(component, manageData=True)

I know how to get all locations with the new API, but I don't know how to excludeInaccessible. Also, the Location.add_component method in the new API does not have a manageData parameter, and also adds a source parameter, so I don't know if it would work the same way.

[Mattias Lagergren]

You're right, those options does not exist but can be implemented fairly easy. 

There is a private method on the Session, Session._get_locations that accepts a filter_inaccessible parameter. Being private it is not something we guarantee will remain the same and the method may change without notice. However, you can look at the implementation here https://bitbucket.org/ftrack/ftrack-python-api/src/4d3f74c47a85492273b251e165f604f579c31f71/source/ftrack_api/session.py?at=master&fileviewer=file-view-default#session.py-1660

As for the manage data parameter it is not supported and is instead determined by the location itself.  If you don't want a location to be managed you can use the UnmanagedLocationMixin which replaces _add_data and _remove_data with empty functions. Our un managed locations are constructed like this:

ftrack_api.mixin(
  location, ftrack_api.entity.location.UnmanagedLocationMixin,
  name='UnmanagedLocation'
)

 

[eight]

Thank you! This helps a lot!

[Mike]

I'm running into problems transferring assets between locations using the connect Maya plugin. I have our pipeline set up in 2 locations: one at work and one home.  Each instance of the pipeline has its own location plugin- work_location.py is in the work pipeline and a home_location.py is in the home pipeline.

Steps:

1. I publish an asset from location A. 
2. When I get to location B, if I try to transfer the asset via the import panel in the Maya plugin I get this error:

# Traceback (most recent call last):
#   File "Z:\GitRepos\_mb_Pipeline\mb_Armada\mb_Armada\modules\dependencies\ftrack_connect\common.zip\ftrack_connect\worker.py", line 54, in run
#     self.result = self.function(*self.args, **self.kwargs)
#   File "Z:\GitRepos\_mb_Pipeline\mb_Armada\mb_Armada\modules\dependencies\ftrack_connect\common.zip\ftrack_connect\ui\widget\component_table.py", line 304, in transfer
#     raise ftrack.FTrackError('Unable to find a suitable source '
# FTrackError: Unable to find a suitable source location to transfer from.

Questions:

1. I'm not quite sure what I'm missing- am I setting everything up and using the plugin correctly, or is the transfer_assets action eight mentioned the correct method to move assets across locations?
2. Should I include both location plugins at location A and B? 

Thanks!

[Mattias Lagergren]

On 9/21/2016 at 0:56 AM, Mike said:

I'm running into problems transferring assets between locations using the connect Maya plugin. I have our pipeline set up in 2 locations: one at work and one home.  Each instance of the pipeline has its own location plugin- work_location.py is in the work pipeline and a home_location.py is in the home pipeline.

For the transfer to work you must have both locations available for transfer. So e.g. if you work at home - location B - you must have location A available to transfer from it. However, you could be representing location A with a different location plugin while at home, e.g. mounted somewhere else or accessed using a FTP.

[Mike]

Thanks for the response!  So how can I tell the Maya ftrack publishing plugin which location to use when publishing?  If I have one location it works fine, but when I add in a 2nd location it breaks because the 2nd location has a different disk prefix.

[Mattias Lagergren]

Quote

 So how can I tell the Maya ftrack publishing plugin which location to use when publishing?

You can set the priority on the location plugin to ensure that the correct location is being picked up.

[Mike]

I have a little time right now, so I'm revisiting locations with the new api. I think I have it sort of working.  

I have a quick question, does ftrack have file transfering across locations built in, or is that up to the developer to create? For example I'm able to see the my current location (home) and the main studio location (work) from the maya import plugin. If i'm at work and I have some files on my home computer, I'm expecting that when I hit transfer in the maya import plugin that it will move the file from my home to computer to my work computer.  This is never the case though. Instead the plugin is just creating the file directory on my local drive with my work's file structure prefix.

Basically, does ftrack have a builtin ability to transfer files over the internet?

 

Current location plugin:
 

[Lucas Correia]

Hi Mike,

File transfers are handled in the API by simply adding the components to the target location:

target_location.add_component(component, source=source_location)

For this to work,  the client needs to have access to both the source and target locations and there is no builtin ability to transfer files over the internet.

There is also an example action available here: Action - transfer components. It can be used to allow transferring of components between multiple locations from the web interface.

Regards,
Lucas

[Mike]

Hi Lucas,

After getting back into locations and taking a look at the docs again, i think accessors are starting to make sense to me. A few questions/comments just to clarify:

1. So ftrack comes with a disk accessor. In my location plugin I've setup my work and home locations with disk accessors. By doing that am I basically telling ftrack that the locations are local?
2. If I want to transfer files over the internet do I need to set my home location to use a disk accessor and my work location to use an internet accessor? (or whatever the accessor would be called)
3. Once those are set, I can use target_location.add_component(component, source=source_location) to easily transfer a file between the two? (as long as both locations are available)

[Lucas Correia]

Hi,

That is correct.

So, what you can do in order to transfer files is to configure the location differently in the different locations.  When on site A, you might have Location A configured with a disk accessor and Location B configured with an accessor communicating over the internet and vice-versa. Note that this is not something we have built-in support for (the API is currently only shipped with a disk accessor), but can be built on top of the locations framework.

Regards,
Lucas

[Mike]

Good to know that I'm understanding it haha.

So how would I go about doing this? I'm looking at the source code and I see base, disk, and server accessors. Could I use the server module as a starting point and the disk accessor as a reference and eventually get some results? What I mean is, is there enough here in the source code that I could eventually figure this out with minimal guidance?

Also, in terms of how to transfer files over the internet, where should I start? Like are there libraries, tutorials, example programs/codes that do this already so I can start to breakdown the process and see exactly what I need? Thanks

[Lucas Correia]

Hi Mike,

To be honest, it does require some development/tooling to get it working well. We don't have any complete examples that does what you are after at this point.

To transfer files across the internet, you would need to write an Accessor which communicates with the other location over an protocol like FTP, rsync, HTTP, etc.. The server accessor is used for our cloud storage and communicates with the server and media server / AWS S3 over HTTP.

The legacy API contained an guide for writing a custom accessor using the Dropbox API, which can be useful to read through to get a grasp of the concepts and what is involved - even though both the ftrack and Dropbox APIs has changed since it was written.

Regards,
Lucas

 

[Mike]

This is great, thanks a lot, Lucas!