Simultaneous ftrack_api.Session() cause lag

[Konstantin Maslyuk]

Hello, i have a case where two ftrack sessions initialized simultaneously from different threads, with overall time spent of 20 seconds, here is profiling result:

         200561 function calls (158003 primitive calls) in 21.062 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   20.759   20.759 c:\Users\kalem\OneDrive\Work\home\dev\test\monorepo\python\inhouse\ftrack\__init__.py:56(__init__)
        1    0.002    0.002   20.758   20.758 C:\Users\kalem\.virtualenvs\test\lib\site-packages\ftrack_api\session.py:76(__init__)
        1    0.001    0.001   12.834   12.834 C:\Users\kalem\.virtualenvs\test\lib\site-packages\ftrack_api\session.py:1405(_load_schemas)
        1    0.636    0.636   12.833   12.833 C:\Users\kalem\.virtualenvs\test\lib\site-packages\ftrack_api\session.py:1365(_read_schemas_from_cache)
        5    0.000    0.000   12.202    2.440 c:\python27\lib\json\__init__.py:193(dumps)
        5    1.557    0.311   12.201    2.440 c:\python27\lib\json\encoder.py:186(encode)
    24534    2.515    0.000   10.643    0.000 c:\python27\lib\json\encoder.py:417(_iterencode)
26576/24534    2.605    0.000    8.129    0.000 c:\python27\lib\json\encoder.py:288(_iterencode_list)
64684/24524    5.341    0.000    5.621    0.000 c:\python27\lib\json\encoder.py:341(_iterencode_dict)

 

[Lorenzo Angeli]

Hi @Konstantin Maslyuk seems most of the time is spent on the json encoder, interesting.
Can you provide what python version are you using and some code to replicate the profiling ?
Thanks.
L.

[Konstantin Maslyuk]

My python version and platform: v2.7.17:c2f86d86e6, win32

You right, i cant reproduce my issue with clean cases:

list(concurrent.futures.ThreadPoolExecutor(3).map(apply, [ftrack_api.Session] * 3))

nither:

pip install git+https://bitbucket.org/ftrack/ftrack-connect-pipeline.git git+https://bitbucket.org/ftrack/ftrack-connect-maya-publish.git https://bitbucket.org/ftrack/qtext/get/0.2.1.zip PySide ftrack-python-legacy-api

python -c "import ftrack_api, pyblish.api, ftrack_connect_pipeline.publish as p, ftrack_connect_pipeline, ftrack_connect_pipeline.application_plugin as g; from QtExt import QtWidgets;import urllib;import logging;logging.basicConfig(level=logging.INFO);G = type('G', (g.BaseApplicationPlugin, ), dict(get_plugin_information=lambda s: {}));a = QtWidgets.QApplication([]);d = {};exec(urllib.urlopen('https://bitbucket.org/ftrack/ftrack-connect-maya-publish/raw/808a548e1a4f409890c864f3234a7381bdd1b590/resource/application_hook/register_scene_asset.py').read(), d);g=G('68e9a0ac-355d-11ea-8ce6-000c29ff458b');import ftrack_connect_pipeline.shared_pyblish_plugins;ftrack_connect_pipeline.shared_pyblish_plugins.register(); pyblish.api.register_plugin(type('Y', (pyblish.api.ContextPlugin, ), dict(order=pyblish.api.CollectorOrder+0.01, process=lambda self, context: context.data.update(dict(ftrack_entity=ftrack_api.Session().get('Context', context.data['ftrack_entity']['id'])))))); d['register'](g.api_session); g.register_assets();ftrack_connect_pipeline.register_plugin(g);p.Publish(g).open()"

(that is a midnight fun). I will investigate my modifications to ftrack_connect_pipeline.

My issue occur when user clicks on workflow, it is added to recent actions in async way that cause (with my modification) new session to be created, then pyblish cycle create session to renew current entity.

[Konstantin Maslyuk]

@Lorenzo Angeli finally i still think that schema loading code is not optimized, a lot of unnecessary work occur inside single process. I could suggest pull request for caching schema.

[Lorenzo Angeli]

hi @Konstantin Maslyuk please feel free to fork the api and provide a PR against our repo ! 
more than interested to look into it.

Thanks!

[Lorenzo Angeli]

Hi @Konstantin Maslyuk just saw the PR , thank for it.
We'll review whenever possible .
Cheers!
L.