ChunkedEncodingError for /documents/versions endpoint

[Michael]

Hey there!
I currently have an issue when I try to POST a new version of a document via the API. This is the code I’m using:

def post_document_versions(
        self, file_name: str, file, document_id: int, comment: str = "new version"
    ) -> None:
        url = urljoin(
            MAYAN_API_HOST, f"api{MAYAN_API_VERSION}/documents/{document_id}/versions/"
        )
        response = requests.post(
            url,
            auth=(MAYAN_API_BASIC_AUTH_USERNAME, MAYAN_API_BASIC_AUTH_PASSWORD),
            data={"active": True, "comment": comment},
            files={"file": (file_name, file, "application/pdf")},
            timeout=10,
        )
        response.raise_for_status()

And this is the resulting error:

def generate():
        # Special case for urllib3.
        if hasattr(self.raw, "stream"):
            try:
>               yield from self.raw.stream(chunk_size, decode_content=True)

.venv/lib/python3.12/site-packages/requests/models.py:816: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.12/site-packages/urllib3/response.py:1040: in stream
    yield from self.read_chunked(amt, decode_content=decode_content)
.venv/lib/python3.12/site-packages/urllib3/response.py:1184: in read_chunked
    self._update_chunk_length()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <urllib3.response.HTTPResponse object at 0x78b8e28d9ab0>

    def _update_chunk_length(self) -> None:
        # First, we'll figure out length of a chunk and then
        # we'll try to read it from socket.
        if self.chunk_left is not None:
            return None
        line = self._fp.fp.readline()  # type: ignore[union-attr]
        line = line.split(b";", 1)[0]
        try:
            self.chunk_left = int(line, 16)
        except ValueError:
            self.close()
            if line:
                # Invalid chunked protocol response, abort.
                raise InvalidChunkLength(self, line) from None
            else:
                # Truncated at start of next chunk
>               raise ProtocolError("Response ended prematurely") from None
E               urllib3.exceptions.ProtocolError: Response ended prematurely

.venv/lib/python3.12/site-packages/urllib3/response.py:1119: ProtocolError

During handling of the above exception, another exception occurred:

self = <src.apps.api.test_mayan.MayanDocumentTest testMethod=test_post_document_versions>

    def test_post_document_versions(self):
        mayan = Mayan()
        document = mayan.get_document_by_label(label=self.label_01)
    
>       mayan.post_document_versions(
            file_name=self.file_name, file=self.file, document_id=document.id
        )

src/apps/api/test_mayan.py:154: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/apps/api/mayan.py:239: in post_document_versions
    response = requests.post(
.venv/lib/python3.12/site-packages/requests/api.py:115: in post
    return request("post", url, data=data, json=json, **kwargs)
.venv/lib/python3.12/site-packages/requests/api.py:59: in request
    return session.request(method=method, url=url, **kwargs)
.venv/lib/python3.12/site-packages/requests/sessions.py:589: in request
    resp = self.send(prep, **send_kwargs)
.venv/lib/python3.12/site-packages/requests/sessions.py:747: in send
    r.content
.venv/lib/python3.12/site-packages/requests/models.py:899: in content
    self._content = b"".join(self.iter_content(CONTENT_CHUNK_SIZE)) or b""
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    def generate():
        # Special case for urllib3.
        if hasattr(self.raw, "stream"):
            try:
                yield from self.raw.stream(chunk_size, decode_content=True)
            except ProtocolError as e:
>               raise ChunkedEncodingError(e)
E               requests.exceptions.ChunkedEncodingError: Response ended prematurely

.venv/lib/python3.12/site-packages/requests/models.py:818: ChunkedEncodingError

I also tried it via curl / postman. Which yields the same results.
Interesting side note: when I curl and use an url with trailing slash the API returns a 404 response. Without a trailing slash it just returns a 200 response but nothing happens.

I can’t find any logs to that in mayan itself and I also added the sentry integration but with no luck.

This issue occures in mayan versions 4.7 - 4.8.2

[Michael]

I digged in the source code and found out that to create new versions you apparently have to use the /documents//files endpoint. I unfortunately receive the same error for that one.

[dbayer]

I’m not 100% sure about this. But I think you need to create the version in one API call, and then upload the file in a second API call.

[roberto.rosario]

You are using the wrong API endpoint in the first instance and using the second API the wrong way.

The endpoint /documents/{}/versions/ is to create a new version not to upload new files. Versions are not files.

The endpoint /documents/{}/files/ can be used to upload new files but only to existing documents.

To upload and create a new document as a single action use the endpoint: /documents/upload/

Or the action API of a source.

This API endpoint was added in version 4.5.

https://docs.mayan-edms.com/releases/4.5.html#sources

You can also use the source actions to upload document in “immediate” mode.

Also always add the trailing slash character, Django expects this and this is what the HTTP standard specifies.

Try getting your API client working using a basic sequential upload before attempting a more complex chunked upload.

You don’t need to specify the content type or the filename for uploads. Use the provided examples as a starting point.

https://docs.mayan-edms.com/apps/rest_api/index.html#examples

[Michael]

@roberto.rosario Thanks for the clarification! And thank you very much for the examples!