Race Condition with Workflows when uploading documents via API

When things don't work as they should.
Post Reply
adzzz
Posts: 2
Joined: Tue Dec 29, 2020 2:34 pm

Race Condition with Workflows when uploading documents via API

Post by adzzz »

Hi,

I've encountered a strange issue where, if a large number of documents are uploaded to Mayan via the REST API, then documents are added to Workflows after a delay (I assume the document_states worker is iterating through these and assigning them to workflows).

The issue is, if events that trigger transitions inside those workflows have completed BEFORE the document was assigned to a specific workflow, then that transition will never trigger. We are using workflow actions to signal back to our application that an OCR event has completed using a HTTP Post callback.

To work around this issue, I thought perhaps we could use the Workflow to trigger the processing of events such as OCR so that they should always occur after a document has been assigned to a workflow:

We did this by setting up two states, an OCR Processing State (initial) and an OCR Complete State. Documents entering the OCR Processing state will trigger an action that calls back to the Mayan API (http://localhost:8000/api/documents/{{d ... cr/submit/), assigning this document to be OCR'd and then subsequently on the OCR complete event it will transition to OCR Complete.

Unfortunately, this API call appears to actually trigger too quickly... only 20-30% of the documents will successfully have been queued for OCR this way, whilst the rest for whatever reason, wont return a status of 1. I assume that this is likely because mayan is still in the process of creating the document when this initial API call is made back onto itself.

My question is, is it possible to delay the action (HTTP Post) by 20 or 30 seconds using the condition block? Similar to a sleep() of sorts to give everything a chance to catch up?

Thank you!
adzzz
Posts: 2
Joined: Tue Dec 29, 2020 2:34 pm

Re: Race Condition with Workflows when uploading documents via API

Post by adzzz »

Anyone there who might be able to lend a hand to assist?
Post Reply