Jump to content

New API: Best practices- caching data for later use.


Recommended Posts

So I'm trying to create a tool that creates tasks from Maya.  I got the initial prototype working, but I'm having trouble with optimization.

Originally, when the user hit the create task button, the session would query the schema, get the available task types, and then put them into a combobox.  This worked fine, but the long loading time proved to be too slow for production.  So I figured what if I loaded the task types when the user first ran the tool? That way when they went to create a task, the combobox would be filled and ready to go.

This worked at first, but along the way things started to fall apart as I added more automation to the code.  Maya started to hang at random times and eventually started to consistently hang during every startup.   I ended up removing the code that builds the combobox data cache and that fixed the issue (so that code is the problem).  The data is being added in a way so the task type name is displayed, but the task type id data is returned on selection.

Here's what I have so far, the problem area is the for loop:

# Ftrack session data to get shot name for tab widget
shot_id = os.getenv('FTRACK_SHOTID')
shot = session.get('Shot', shot_id)
self.shot_name = shot['name']
project = shot['project']
schema = project['project_schema']
 
self.cb_task_types = QtGui.QComboBox()
self.task_types = schema.get_types('Task')
 
for task_type in self.task_types:
	self.cb_task_types.addItem(task_type['name'], task_type['id'])

 

Edited by Mattias Lagergren
Fixed code highlighting issue
Link to post
Share on other sites

Hi Mike,

The ftrack-python-api already comes with caching already built-in. All data that is fetched from the server is cached in-memory. 

I've taken your example and modified it to be more performant (I haven't tested it so there may be syntax errors):

# Ftrack session data to get shot name for tab widget
shot_id = os.getenv('FTRACK_SHOTID')

# Load shot and related data in one query. This avoids fetching name and
# project schema separately.
shot = session.query(
    'select name, project.project_schema from Shot where id is "{0}"'.format(
        shot_id
    )
).one()

self.shot_name = shot['name']
project = shot['project']
schema = project['project_schema']
 
self.cb_task_types = QtGui.QComboBox()
self.task_types = schema.get_types('Task')

# Pre-fetch all type names so that they are cached. This avoids a query
# per type in the loop below.
session.query('select name from Type').all()

for task_type in self.task_types:
    self.cb_task_types.addItem(task_type['name'], task_type['id'])

 

Link to post
Share on other sites

Just a quick question also:

So if I run your edited code in my app's main window code, does that mean I can grab that cached data from another module just by querying again?

For example, should I be querying the shot like this in every module:

self.session = ftrack_api.Session()

shot_id = os.getenv('FTRACK_SHOTID')

shot = self.session.query(
	'select name, from Shot where id is "{0}"'.format(
		shot_id
	)
).one()

Or is there another intended way to do this?

Link to post
Share on other sites
16 hours ago, Mike said:

So if I run your edited code in my app's main window code, does that mean I can grab that cached data from another module just by querying again?

The cache is kept by the session, in memory. So if you want to use the cache you must share the session between your modules.

A common way could be to:

import my_helper_module

# Get the shared session where my module creates the session
# once and then re-uses it.
session = my_helper_module.get_shared_session()

 

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...