MAYAN_APT_INSTALLS does not work for Archlinux with kernels > 4.14

When things doesn't work as they should.
Harvey
Posts: 11
Joined: Tue Oct 30, 2018 12:03 pm

MAYAN_APT_INSTALLS does not work for Archlinux with kernels > 4.14

Post by Harvey » Tue Oct 30, 2018 12:34 pm

Hello @all,

this is my first post here so don't beat me if I do something wrong :D

I try to evaluate Mayan EDMS for our non-profit-organisation (a home for mentally affected people).
As we are in Germany I try to install the german tesseract files tesseract-ocr-deu tesseract-ocr-deu-frak via MAYAN_APT_INSTALLS. My complete command line looks like this:

Code: Select all

docker run -d --name mayan-edms --restart=always -p 81:8000 \
-e MAYAN_DATABASE_ENGINE=django.db.backends.postgresql \
-e MAYAN_DATABASE_HOST=172.17.0.1 \
-e MAYAN_DATABASE_NAME=mayan \
-e MAYAN_DATABASE_PASSWORD=mysupersecret \
-e MAYAN_DATABASE_USER=mayan \
-e MAYAN_DATABASE_CONN_MAX_AGE=60 \
-e MAYAN_APT_INSTALLS="tesseract-ocr-deu tesseract-ocr-deu-frak" \
-v /home/docker-volumes/mayan-edms/media:/var/lib/mayan \
-v /srv/scanned-for-EDMS:/srv/watch_folder \
mayanedms/mayanedms:latest
The container starts and I start a shell in there to evaluate if all is going well.

Code: Select all

docker exec -i -t mayan-edms /bin/bash
In the shell I then look for tesseract files:

Code: Select all

root@4eeb966c15b2:/opt/mayan-edms# apt list --installed tesseract*
Listing... Done
tesseract-ocr/stable,now 3.04.01-5 amd64 [installed]
tesseract-ocr-eng/stable,now 3.04.00-1 all [installed]
tesseract-ocr-equ/stable,now 3.04.00-1 all [installed]
tesseract-ocr-osd/stable,now 3.04.00-1 all [installed] 
No installation of tesseract-ocr-deu tesseract-ocr-deu-frak, right?
I did some forensics then:

Code: Select all

root@4eeb966c15b2:/opt/mayan-edms# apt update
Hit:1 http://security.debian.org/debian-security stretch/updates InRelease
Ign:2 http://cdn-fastly.deb.debian.org/debian stretch InRelease           
Hit:3 http://cdn-fastly.deb.debian.org/debian stretch-updates InRelease
Hit:4 http://cdn-fastly.deb.debian.org/debian stretch Release
Reading package lists... Done
Building dependency tree       
Reading state information... Done
4 packages can be upgraded. Run 'apt list --upgradable' to see them.

root@4eeb966c15b2:/opt/mayan-edms# apt install tesseract-ocr-deu
E: dpkg was interrupted, you must manually run 'dpkg --configure -a' to correct the problem. 

root@4eeb966c15b2:/opt/mayan-edms# dpkg --configure -a
dpkg: error: error creating new backup file '/var/lib/dpkg/status-old': Invalid cross-device link
root@4eeb966c15b2:/opt/mayan-edms# 
There is definitely something wrong here. Is the error on my side or is it the docker image?

Greetings
Harvey
Last edited by Harvey on Sat Nov 10, 2018 2:22 pm, edited 1 time in total.

User avatar
rosarior
Posts: 159
Joined: Tue Aug 21, 2018 3:28 am

Re: MAYAN_APT_INSTALLS does not work for 3.1.7 docker image

Post by rosarior » Wed Oct 31, 2018 8:07 pm

Hi, thanks for the report.

Looks like the installation of the tesseract packages was not allowed to finish

Code: Select all

E: dpkg was interrupted, 
and the package database is in a corrupted states requiring additional command to fix it.

Code: Select all

you must manually run 'dpkg --configure -a' to correct the problem. 
I would recommend deleting the container and launching it again, containers are meant to be ephemeral. The data will not be deleted if the volumes containing it are not removed. Same goes for the database if the database container volumes are also not removed, (but for this there is no need to remove the database container).

Harvey
Posts: 11
Joined: Tue Oct 30, 2018 12:03 pm

Re: MAYAN_APT_INSTALLS does not work for 3.1.7 docker image

Post by Harvey » Fri Nov 02, 2018 10:35 am

rosario,

thank you for your patience.

I did the whole installation once again, starting with a completely new installation of docker to rule out every side-effects. I followed the chapter about docker installation thoroughly but the installation of additional tesseract-data files did not work. :(

Again when starting a shell in the container there are only the standard files installed and the dpkg database is corrupted:

Code: Select all

root@318ce21a1e5f:/opt/mayan-edms# dpkg --configure -a
dpkg: error: error creating new backup file '/var/lib/dpkg/status-old': Invalid cross-device link
I guess this corruption is already in the image, because installing the container without using MAYAN_APT_INSTALLS is also resulting in the same error.
The shell inside the container says:

Code: Select all

root@97e477a5d7e3:/opt/mayan-edms# apt list --installed tesser*
Listing... Done
tesseract-ocr/now 3.04.01-5 amd64 [installed,local]
tesseract-ocr-eng/now 3.04.00-1 all [installed,local]
tesseract-ocr-equ/now 3.04.00-1 all [installed,local]
tesseract-ocr-osd/now 3.04.00-1 all [installed,local]
root@97e477a5d7e3:/opt/mayan-edms# apt update
Get:1 http://security.debian.org/debian-security stretch/updates InRelease [94.3 kB]
Ign:2 http://cdn-fastly.deb.debian.org/debian stretch InRelease                          
Get:3 http://cdn-fastly.deb.debian.org/debian stretch-updates InRelease [91.0 kB]        
Get:4 http://cdn-fastly.deb.debian.org/debian stretch Release [118 kB]             
Get:5 http://cdn-fastly.deb.debian.org/debian stretch Release.gpg [2434 B]
Get:6 http://security.debian.org/debian-security stretch/updates/main amd64 Packages [454 kB]
Get:7 http://cdn-fastly.deb.debian.org/debian stretch-updates/main amd64 Packages [5152 B]
Get:8 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 Packages [7099 kB] 
Fetched 7864 kB in 3s (2587 kB/s)   
Reading package lists... Done
Building dependency tree... Done
4 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@97e477a5d7e3:/opt/mayan-edms# apt list --upgradable
Listing... Done
base-files/stable 9.9+deb9u5 amd64 [upgradable from: 9.9+deb9u4]
dpkg/stable 1.18.25 amd64 [upgradable from: 1.18.24]
libsystemd0/stable 232-25+deb9u4 amd64 [upgradable from: 232-25+deb9u3]
tzdata/stable-updates 2018g-0+deb9u1 all [upgradable from: 2018e-0+deb9u1]
root@97e477a5d7e3:/opt/mayan-edms# apt install tesseract-ocr-deu
Reading package lists... Done
Building dependency tree... Done
The following NEW packages will be installed:
  tesseract-ocr-deu
0 upgraded, 1 newly installed, 0 to remove and 4 not upgraded.
Need to get 4153 kB of archives.
After this operation, 13.4 MB of additional disk space will be used.
Get:1 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 tesseract-ocr-deu all 3.04.00-1 [4153 kB]
Fetched 4153 kB in 0s (4410 kB/s)            
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package tesseract-ocr-deu.
(Reading database ... 31136 files and directories currently installed.)
Preparing to unpack .../tesseract-ocr-deu_3.04.00-1_all.deb ...
Unpacking tesseract-ocr-deu (3.04.00-1) ...
dpkg: error: error creating new backup file '/var/lib/dpkg/status-old': Invalid cross-device link
E: Sub-process /usr/bin/dpkg returned an error code (2)
Again: this is with a new container. I guess the main problem is this message:

Code: Select all

dpkg: error: error creating new backup file '/var/lib/dpkg/status-old': Invalid cross-device link
dpkg is not able to create this file and hence fails. I guess this is exactly the same what happens when using MAYAN_APT_INSTALLS.
In my opinion, the docker image is at fault here.

Greetings
Harvey

Harvey
Posts: 11
Joined: Tue Oct 30, 2018 12:03 pm

Re: MAYAN_APT_INSTALLS does not work for 3.1.7 docker image

Post by Harvey » Tue Nov 06, 2018 1:56 pm

Really no ideas?

Greetings
Harvey

KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Re: MAYAN_APT_INSTALLS does not work for 3.1.7 docker image

Post by KevinPawsey » Fri Nov 09, 2018 12:38 pm

Have you tried with the 3.1.9 image? I have recently installed the 3.1.9 image on AMD64, and have not had any issues with the image itself.

I haven't tried to install any extra libraries on the standard install... but are the quotes (") needed around the extra tesseract libraries to be installed?

Sorry I can't be any more specific with any other help.


Kevin

User avatar
rosarior
Posts: 159
Joined: Tue Aug 21, 2018 3:28 am

Re: MAYAN_APT_INSTALLS does not work for 3.1.7 docker image

Post by rosarior » Fri Nov 09, 2018 3:46 pm

Have not been able to replicate this on my systems or in Play With Docker.

Appears to be an issue with the overlay driver on older kernels: https://bugs.debian.org/cgi-bin/bugrepo ... bug=836211

Make sure the host of is running a recent kernel, distribution and version of Docker.
Attachments
Mayan Docker APT_INSTALLS.png
Mayan Docker APT_INSTALLS.png (85.56 KiB) Viewed 179 times

Harvey
Posts: 11
Joined: Tue Oct 30, 2018 12:03 pm

Re: MAYAN_APT_INSTALLS does not work for Archlinux with kernels > 4.14

Post by Harvey » Sat Nov 10, 2018 3:06 pm

Thank you both for your input. I have done some more research and here is what I got meanwhile:

I am on Archlinux with kernel 4.19.1 using the most recent docker version. So older kernels and docker software are not the culprit here.

I thought it might be the other way round because all the other distributions use older kernels. So I installed the Archlinux LTS kernel instead (4.14.79) and tried again.

What should I say? The german tesseract files install o.k. using the same docker installation!

So there has to be a difference between 4.14 and 4.19 that causes my problems. I found this one
https://bugs.archlinux.org/task/57336 but simply starting 4.19.1 with the kernel parameter vsyscall=emulate did not do the job.

Will have to dig deeper into this.

Greetings
Harvey

Harvey
Posts: 11
Joined: Tue Oct 30, 2018 12:03 pm

Re: MAYAN_APT_INSTALLS does not work for Archlinux with kernels > 4.14

Post by Harvey » Tue Nov 13, 2018 10:20 am

Just for the records: it is a problem of kernel 4.19 not playing well with docker's overlayfs. There is already a bugreport.

KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Re: MAYAN_APT_INSTALLS does not work for Archlinux with kernels > 4.14

Post by KevinPawsey » Tue Nov 13, 2018 12:00 pm

Ah, cool, thanks for letting us know... although I don't use Archlinux myself (I use OpenMediaVault, which is Debian-based), if it is a kernel thing I will avoid going to 4.19 if they bring that out... just in case :)

User avatar
rosarior
Posts: 159
Joined: Tue Aug 21, 2018 3:28 am

Re: MAYAN_APT_INSTALLS does not work for Archlinux with kernels > 4.14

Post by rosarior » Thu Nov 15, 2018 6:16 am

Thanks for updating the title of the topic, helps others having the same issue.
Also thanks for the links to the upstream issues, will be easier to track their progress.

Post Reply