New API: Best practices- caching data for later use.

[Mike]

So I'm trying to create a tool that creates tasks from Maya.  I got the initial prototype working, but I'm having trouble with optimization.

Originally, when the user hit the create task button, the session would query the schema, get the available task types, and then put them into a combobox.  This worked fine, but the long loading time proved to be too slow for production.  So I figured what if I loaded the task types when the user first ran the tool? That way when they went to create a task, the combobox would be filled and ready to go.

This worked at first, but along the way things started to fall apart as I added more automation to the code.  Maya started to hang at random times and eventually started to consistently hang during every startup.   I ended up removing the code that builds the combobox data cache and that fixed the issue (so that code is the problem).  The data is being added in a way so the task type name is displayed, but the task type id data is returned on selection.

Here's what I have so far, the problem area is the for loop:

# Ftrack session data to get shot name for tab widget
shot_id = os.getenv('FTRACK_SHOTID')
shot = session.get('Shot', shot_id)
self.shot_name = shot['name']
project = shot['project']
schema = project['project_schema']
 
self.cb_task_types = QtGui.QComboBox()
self.task_types = schema.get_types('Task')
 
for task_type in self.task_types:
	self.cb_task_types.addItem(task_type['name'], task_type['id'])

 

[Mattias Lagergren]

Hi Mike,

The ftrack-python-api already comes with caching already built-in. All data that is fetched from the server is cached in-memory. 

I've taken your example and modified it to be more performant (I haven't tested it so there may be syntax errors):

# Ftrack session data to get shot name for tab widget
shot_id = os.getenv('FTRACK_SHOTID')

# Load shot and related data in one query. This avoids fetching name and
# project schema separately.
shot = session.query(
    'select name, project.project_schema from Shot where id is "{0}"'.format(
        shot_id
    )
).one()

self.shot_name = shot['name']
project = shot['project']
schema = project['project_schema']
 
self.cb_task_types = QtGui.QComboBox()
self.task_types = schema.get_types('Task')

# Pre-fetch all type names so that they are cached. This avoids a query
# per type in the loop below.
session.query('select name from Type').all()

for task_type in self.task_types:
    self.cb_task_types.addItem(task_type['name'], task_type['id'])

 

[Mike]

5 hours ago, Mattias Lagergren said:

 

# Pre-fetch all type names so that they are cached. This avoids a query
# per type in the loop below.
session.query('select name from Type').all()

Perfect, that's what I needed!

 

[Mike]

Just a quick question also:

So if I run your edited code in my app's main window code, does that mean I can grab that cached data from another module just by querying again?

For example, should I be querying the shot like this in every module:

self.session = ftrack_api.Session()

shot_id = os.getenv('FTRACK_SHOTID')

shot = self.session.query(
	'select name, from Shot where id is "{0}"'.format(
		shot_id
	)
).one()

Or is there another intended way to do this?

[Mattias Lagergren]

16 hours ago, Mike said:

So if I run your edited code in my app's main window code, does that mean I can grab that cached data from another module just by querying again?

The cache is kept by the session, in memory. So if you want to use the cache you must share the session between your modules.

A common way could be to:

import my_helper_module

# Get the shared session where my module creates the session
# once and then re-uses it.
session = my_helper_module.get_shared_session()