Event_hub history

[tdugard]

Hello everyone,

 

I'm trying to use the eventHub to replicate ftrack actions into a local database. Everything is working fine but i have 2 questions :

• It is possible to subscribe to the event_hub with a debug flag to show crashes ? because actually when something went wrong, no exception is raised.
• To make it solid i need to access the event history. Because if my callback program crashes( (internet failure, machine crashes, or code crashes) i need to know what happenned since the last event I had replicated in my local database. So i can process all the missing events when the callback script is running again.

Thanks for reading this,

 

[Nebukadhezer]

I would want to know this too.
With another production management tool the event listener would store the event process id and upon crash it would start working its way up... from the last stored id.
Would sth like this be possible?

 

[Henrik Norin]

Hi,

This is available as an advanced configuration option for ftrack on-prem deployments:

http://help.ftrack.com/en/articles/1040523-configuring-server-options#fetch-update-events-directly-from-internal-queue

Cheers

/Henrik

[tdugard]

So it's not possible to have the event history with an online ftrack setup ?

I came from an another production management software where all the events are stored in logs which was really useful. I think this is a must have feature to develop a strong pipeline.

[Henrik Norin]

Hi,

 

Forgot to point out that events also can be queried by the API:

https://bitbucket.org/ftrack/workspace/snippets/ggyrb/api-example-working-with-events

 

Cheers

/Henrik 

[tdugard]

Yes that's what i was looking for ! I can save the last processed event id and then when i reopen my callback i just have to process events that are newer than this id

[tdugard]

I successfully replaced the connection to the eventhub with a custom script querying the ftrack database event using the api but i have one last simple question : 

Actually i am doing 2 requests to the database : 

event = session.query('Event where id is "{}"'.format(event_id)).first()
events = session.query('Event where created_at > "{}"'.format(event['created_at']))

Is it possible to merge it as one query ?

[Guest]

You can do a subquery (bottom of the page here https://help.ftrack.com/en/articles/1040506-query-syntax) which would look like 

'Event where created_at > (select created_at from Event where id is "{}")'

As an aside, also take a look at the section on "projections". The two lines you pasted actually do three requests in the case where the "created_at" attribute wasn't yet cached. To avoid that automatic call, you can request all the attributes you'll need all at once like so:

event = session.query('select created_at from Event where id is "{}"'.format(event_id)).first()
events = session.query('Event where created_at > "{}"'.format(event['created_at']))

[tdugard]

OK i see how to do that, thanks, but when i tried to manage shot and user events doing queries to the database, i found some bugs :

• When i got a "Project' event (insert, remove, update, move) the parent type is a show so i think it's the correct behavior but when i got a "Shot" event (insert, remove, update, move) the parent type is a task ? Why a task ? Maybe i'm doing it wrong to know the entity concerned by the event.
• When i create a user there is no event added to the Events table in ftrack database ! When i used the eventhub callback (my previous solution), user's events (create, delete, update) were triggered.

Is it normal ? 

[Guest]

"show" and "task" are legacy identifiers, roughly corresponding to Project and TypedContext in the new API. session.get('Context', id) should return the most-specific class available for the given it.

In my demo content, I have the following parent types:

appointment
asset
assetversion
asset_version
dependency
list
note
review_session
review_session_object
review_session_object_status
show
task

One of the original designers might have to confirm, but I believe that we only persist to the db Events which trigger a notification/will be rendered in the UI. AFIAK, creating a user does not do this.

[tdugard]

The problem is i can't rely on session.get('Context', id) because of the delete operation. When i check a 'delete' event, i can't use that fonction because this entity no longer exists. So i can't check its type when it's a deletion.

I managed with lot's of if/else to identify the entity (actually i don't need to reproduce every table of ftrack db just the main ones). 

Do you planned to rework the event in db so the event hub and event in db are formatted the same way ?

 

That's strange about the users because the eventhub fires an event for user creation/deletion/update. Maybe i can make a second script to watch the user table instead of Events

[tdugard]

Hello, 

i've got a new problem trying to use the event table. When i create shot with a shot template (shot + task) the task are registering event before the shot, here is the output of my tool : 

2020-11-23 18:40:44,118 - [ERROR] [#117319] Can't create the task "FX" <ftrack_id:fafc2d40-2db2-11eb-b45c-6e9c55d3c1f9>, missing parent in database <ftrack_id:ed267a70-2daa-11eb-a478-6e9c55d3c1f9>
2020-11-23 18:40:44,359 - [ERROR] [#117320] Can't create the task "Compositing" <ftrack_id:fafbcb16-2db2-11eb-b45c-6e9c55d3c1f9>, missing parent in database <ftrack_id:ed267a70-2daa-11eb-a478-6e9c55d3c1f9>
2020-11-23 18:40:44,631 - [ERROR] [#117321] Can't create the task "Cloth" <ftrack_id:fafb6400-2db2-11eb-b45c-6e9c55d3c1f9>, missing parent in database <ftrack_id:ed267a70-2daa-11eb-a478-6e9c55d3c1f9>
2020-11-23 18:40:44,891 - [INFO] [#117322] Shot "test_project/3000" created <id:5fbbf41cac16c5229f32aa67, ftrack_id:faeffd18-2db2-11eb-b45c-6e9c55d3c1f9>
2020-11-23 18:40:45,233 - [INFO] [#117323] Task "test_project/3000/Animation" created <id:5fbbf41dac16c5229f32aa68, ftrack_id:fafac54a-2db2-11eb-b45c-6e9c55d3c1f9>
2020-11-23 18:40:45,551 - [INFO] [#117324] Task "test_project/3000/Lighting" created <id:5fbbf41dac16c5229f32aa69, ftrack_id:fafe5f84-2db2-11eb-b45c-6e9c55d3c1f9>
2020-11-23 18:40:45,892 - [INFO] [#117325] Task "test_project/3000/Post_Animation" created <id:5fbbf41dac16c5229f32aa6a, ftrack_id:faf84144-2db2-11eb-b45c-6e9c55d3c1f9>
2020-11-23 18:40:46,220 - [INFO] [#117326] Task "test_project/3000/Matte_Painting" created <id:5fbbf41eac16c5229f32aa6b, ftrack_id:fafd8708-2db2-11eb-b45c-6e9c55d3c1f9>
2020-11-23 18:40:46,608 - [INFO] [#117327] Task "test_project/3000/Mocap" created <id:5fbbf41eac16c5229f32aa6c, ftrack_id:fafd2a60-2db2-11eb-b45c-6e9c55d3c1f9>
2020-11-23 18:40:46,944 - [INFO] [#117328] Task "test_project/3000/Assembly" created <id:5fbbf41eac16c5229f32aa6d, ftrack_id:fafdea18-2db2-11eb-b45c-6e9c55d3c1f9>

You can see the first 3 events : 117319, 117320, 117321 are tasks related to the shot created at event 117322. All remaining task (event 117323-117328) are cloned correcty. 

[tdugard]

I managed to do it another way not using the event but just the ftrack id of the entity and recurse creation of its ancestors. But last thing the : 

session.get('Context', id)

returns a correct value for Project, Task, Shot, AssetBuild but return None for Note, AssetVersion, Asset how can i know exactly the type of object with only the id ?

[tdugard]

Again me, i add to the previous question : 

Is there a list somewhere of all event action like 'change.status.task' because at the moment I need to test by hand every change to know every event action possible ?

[Henrik Norin]

FYI, this has been discussed and explained in a separate thread:  https://forum.ftrack.com/topic/677-sessiongettypedcontext-entity_id-not-working/?tab=comments#comment-3584