Scanning as input

When things doesn't work as they should.
Post Reply
KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Scanning as input

Post by KevinPawsey » Fri Nov 02, 2018 10:42 pm

Hi there,

well... I have bitten the bullet and decided to resurrect an old workstation for my Mayan-EDMS server... at least it is an AMD64-based machine, so getting the docker image isn't an issue :) ... plus I can do some basic (RAID1) mirroring in hardware with it, so I wont lose my documents in a hurry ;)

Anyway... since rebuilding my EDMS server, I have a couple of issues... scanning is one of them.

I never got scanning to work when I was running on my Rock64, but then I never really tried that hard... but now that I have this as a 'permanent' solution, I am not thinking that I should be able to get this working.

I have setup the scanning source, and filled out the scan source as follows:

Label: Brother
I am assuming this is just identifying when you use the input

Device name: `brother4:net1;dev0'
I got the device name from the following:

Code: Select all

root@mayan-edms:~/brscan# scanimage -L
device `brother4:net1;dev0' is a Brother *DCP7065DN DCP7065DN
device `brother4:bus2;dev1' is a Brother DCP-7065DN USB scanner
I have tried with and without the ' and also I have had a look at the output of

Code: Select all

brsaneconfig4 -q
and that produces a LONG list of supported printers with

Code: Select all

Devices on network
  0 DCP7065DN           "DCP7065DN"         I:192.168.1.83
  
at the end.

I have also tried this in the Device name field... however, whenever I try and scan, I see the following in the logs:

Code: Select all

sources.models <86> [ERROR] "get_upload_file_object() line 266 Exception while scanning from source:Brother ; scanimage: open of device brother4:net1;dev0 failed: Invalid argument
"
I have also tried the USB as well as the network device (yes, it is the same printer on both network and USB), but it is always the same thing.

If I try and run:

Code: Select all

root@mayan-edms:~/brscan# brscan-skey 
root@mayan-edms:~/brscan# brscan-skey -l

 DCP7065DN         : brother4:net1;dev0  : 192.168.1.83         Active
 DCP-7065DN        : brother4:bus2;dev1  : USB                  Active
then if I go to the printer and scan to PC... it works perfectly... I have even thought about just setting up the folder that it scans to as a watch folder... then at least I could scan stuff in :)

Have I missed something somewhere? Am I not getting the device name from the right place?

Any help is greatly appreciated... now onto my other outstanding issue... in another thread :)

Thanks in advance

Kevin

User avatar
rosarior
Posts: 159
Joined: Tue Aug 21, 2018 3:28 am

Re: Scanning as input

Post by rosarior » Mon Nov 05, 2018 5:38 am

I'm using an EPSON multifunction printer and it works ok. Out of curiosity tested another scanner source using an HP multifunction printer and got the same error. There is nothing different on how they are called using the scanimage binary. The only difference I can see is that scanner resource line returned by scanimage for my HP has a "?" in it and yours has a ";". My suspicion is that some characters are not being encoded properly when scanimage is being called internally. I've added this for further research.

KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Re: Scanning as input

Post by KevinPawsey » Mon Nov 05, 2018 3:11 pm

Appreciate you to taking a look at this... I will have a further play to see if I can get anything to happen on the cli and will pass on anything that I find.

Kevin

KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Re: Scanning as input

Post by KevinPawsey » Mon Nov 05, 2018 3:51 pm

I had a lightbulb moment as I finished typing out my previous reply...

Being that I am using Docker I realised that the scanimage -L results were from OUTSIDE the Docker container! ... DOH!

Then I realised that this means that to get it working I have to do the following:
  • install cups
  • install brother drivers
... this means that each time the image is upgraded I would need to perform this :(

I thought that to at least get this working I would give it a try to make sure that this is all that the issues were... I installed everything, and I can indeed now see the scanner from within the Docker container with the scanimage -L command. So this is (kind of) sorted... the issue that I have at the moment is the fact that scanimage -L works, whereas any other commands in scanimage are throwing a segmentation fault:

Code: Select all

root@mayan-edms:/var/lib/mayan# scanimage --device-name="brother4:net1;dev0"
Segmentation fault (core dumped)
I think that this is something to do with the brother drivers... so I will have a look into how this might be able to be solved.

Failing that... do you have a model number of the Epson scanner that you have working... maybe I should just invest in a decent, and properly supported scanner.

Kevin

KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Re: Scanning as input

Post by KevinPawsey » Mon Nov 05, 2018 4:25 pm

OK.. think I may have got to the bottom of it...

I rebooted the Docker and (for some reason) lost all the CUPS and brother drivers... *sigh*

I ran sane-find-scanner to see if I could see the scanner at all within the Docker image, and it returned:

Code: Select all

root@mayan-edms:/var/lib/mayan# sane-find-scanner 

  # sane-find-scanner will now attempt to detect your scanner. If the
  # result is different from what you expected, first make sure your
  # scanner is powered up and properly connected to your computer.

  # No SCSI scanners found. If you expected something different, make sure that
  # you have loaded a kernel SCSI driver for your SCSI adapter.

could not open USB device 0x1d6b/0x0002 at 008:001: No such device (it may have been disconnected)
could not open USB device 0x1d6b/0x0001 at 006:001: No such device (it may have been disconnected)
could not open USB device 0x1d6b/0x0001 at 005:001: No such device (it may have been disconnected)
could not open USB device 0x1d6b/0x0001 at 004:001: No such device (it may have been disconnected)
could not open USB device 0x04f9/0x024a at 007:002: No such device (it may have been disconnected)
could not open USB device 0x1d6b/0x0002 at 007:001: No such device (it may have been disconnected)
could not open USB device 0x1d6b/0x0001 at 003:001: No such device (it may have been disconnected)
could not open USB device 0x1d6b/0x0001 at 002:001: No such device (it may have been disconnected)
could not open USB device 0x1d6b/0x0001 at 001:001: No such device (it may have been disconnected)
  # No USB scanners found. If you expected something different, make sure that
  # you have loaded a kernel driver for your USB host controller and have setup
  # the USB system correctly. See man sane-usb for details.

  # Not checking for parallel port scanners.

  # Most Scanners connected to the parallel port or other proprietary ports
  # can't be detected by this program.
So I assumed that the USB ports were not exposed to the Docker container... so I changed the option in Docker to run as Privileged, and restarted.

On restarting the sane-find-scanner command output changed:

Code: Select all

root@mayan-edms:/opt/mayan-edms# scanimage -L

No scanners were identified. If you were expecting something different,
check that the scanner is plugged in, turned on and detected by the
sane-find-scanner tool (if appropriate). Please read the documentation
which came with this software (README, FAQ, manpages).
root@mayan-edms:/opt/mayan-edms# sane-find-scanner 

  # sane-find-scanner will now attempt to detect your scanner. If the
  # result is different from what you expected, first make sure your
  # scanner is powered up and properly connected to your computer.

  # No SCSI scanners found. If you expected something different, make sure that
  # you have loaded a kernel SCSI driver for your SCSI adapter.

[b]found USB scanner (vendor=0x04f9, product=0x024a) at libusb:007:002[/b]
  # Your USB scanner was (probably) detected. It may or may not be supported by
  # SANE. Try scanimage -L and read the backend's manpage.

  # Not checking for parallel port scanners.

  # Most Scanners connected to the parallel port or other proprietary ports
  # can't be detected by this program.
So it could actually now see the scanner within the Docker container now (step forward!!!).

The results of scanimage -L were still not showing a scanner, but this was to be expected, as this happened before when I was trying to get it to work... so I reinstalled the scanner driver (note: not the entire printer driver suite and CUPS):

Code: Select all

root@mayan-edms:/var/lib/mayan# dpkg -i --force-all brscan4-0.4.6-1.amd64.deb 
Selecting previously unselected package brscan4.
(Reading database ... 31136 files and directories currently installed.)
Preparing to unpack brscan4-0.4.6-1.amd64.deb ...
Unpacking brscan4 (0.4.6-1) ...
Setting up brscan4 (0.4.6-1) ...
This software is based in part on the work of the Independent JPEG Group.
-e WARNING: libusb-0.1 is required.
The I tried running scanimage -L again:

Code: Select all

root@mayan-edms:/var/lib/mayan# scanimage -L
device `brother4:bus7;dev5' is a Brother DCP-7065DN USB scanner
WOOHOO :lol:

I then went to the source config, put in brother4:bus7;dev5 as being the device name, and now it scans!! :D

Now... a couple of things:
  • should mayan-edms need to be run in privileged if you are using a USB scanner (the DCP-7065DN is actually a network scanner... so I may be able to get round this)
  • is there a way of making the driver install persistent across upgrades of mayan?
Hopefully this has helped anyone else trying to get their scanner working (or am I the only one having issues?? :lol:)

Regards

Kevin

KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Re: Scanning as input

Post by KevinPawsey » Tue Nov 06, 2018 2:36 pm

ok... further information... after a couple of attempts the scanning seemed to stop... which is a bit odd. I did a lot of research, and reinstalled the Brother scanning drivers again... and I noticed that it said that it was needing libusb!

I installed libusb with:

Code: Select all

apt-get install libusb*
from within the container... yes, this is massive overkill for the amount that is probably required... I may just get away with libusb-dev, and looking at the research libusbmuxd may be another one... I will refine this later.

The main thing is... I run the scanimage -L again, and found this:

Code: Select all

root@mayan-edms:/opt/mayan-edms# scanimage -L
device `brother4:net1;dev0' is a Brother *DCP7065DN 
device `brother4:bus2;dev1' is a Brother DCP-7065DN USB scanner
which is a different device from where it was originally... which is a little odd... but I am not complaining, as it seems to be working! ...

... there is a but... but only for single pages. If I put multiple pages in the document feeder I get the following error:

Code: Select all

[2018-11-06 14:15:13,907: ERROR/Worker-1] Error sending data
Traceback (most recent call last):
  File "/opt/mayan-edms/local/lib/python2.7/site-packages/gnupg.py", line 156, in _copy_data
    outstream.write(data)
IOError: [Errno 32] Broken pipe
[2018-11-06 14:15:14,349: ERROR/Worker-1] Error sending data
Traceback (most recent call last):
  File "/opt/mayan-edms/local/lib/python2.7/site-packages/gnupg.py", line 156, in _copy_data
    outstream.write(data)
IOError: [Errno 32] Broken pipe
When it scans, the first page is fine, but then the rest of the pages are just pulled through without scanning, and only the first page is stored.

Anyway, it is all progress... I am considering re-creating my mayan container, starting again from scratch so that I can see exactly all that is needed to get it working. It is looking like an inclusion of at least some of the libusb libraries will be needed though.

User avatar
rosarior
Posts: 159
Joined: Tue Aug 21, 2018 3:28 am

Re: Scanning as input

Post by rosarior » Fri Nov 09, 2018 3:52 pm

Docker containers are meant to be ephemeral. You can store the changes to a container by using Docker's commit command to create a custom image from a running containers: https://stackoverflow.com/questions/242 ... ial-device

In Mayan we provide the MAYAN_APT_INSTALLS to install Debian packages when the containers starts.

Here is Mayan's chapter of customizing Docker installations: https://docs.mayan-edms.com/chapters/do ... -the-image

You can share devices using the --priviledge option or by sharing individual devices: https://stackoverflow.com/questions/242 ... ial-device

User avatar
rosarior
Posts: 159
Joined: Tue Aug 21, 2018 3:28 am

Re: Scanning as input

Post by rosarior » Fri Nov 09, 2018 3:55 pm

USB enumeration is done at runtime and is bound to assign different device numbers. ie: Arduinos when plugged get different serial ports. In these cases a custom udev entry is required to map a device vendor ID to a specific /dev entry.

If the scanner also supports ethernet I just set it to a static IP and avoid the USB issue.

KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Re: Scanning as input

Post by KevinPawsey » Sat Nov 10, 2018 10:27 am

In Mayan we provide the MAYAN_APT_INSTALLS to install Debian packages when the containers starts.
ah yes, I forgot about this... ironically I answered a post to someone else using the same option, I didn’t think of using it to install the USB package.

I believe that the Rock64 assigns the same USB device on each reboot, so I think that this is not an issue. I do agree that the network scanning is going to be a better option (I have a cable strung across the room at the moment!)... so I will test getting that working next :)

I’ll have a read up on customizing the containers... thanks again for your help Rosarior, awesome help and support as always :)

KevinPawsey
Posts: 50
Joined: Wed Aug 22, 2018 2:52 pm

Re: Scanning as input

Post by KevinPawsey » Mon Nov 12, 2018 12:50 pm

rosarior wrote:
Fri Nov 09, 2018 3:55 pm
If the scanner also supports ethernet I just set it to a static IP and avoid the USB issue.
well, I seem to have managed to get the network scanning working... so hopefully this will get round the need for the USB libraries... I just need to install the brother drivers and setup the network scanner on install.

I do still seem to be getting this error after the first page is scanned:

Code: Select all

[2018-11-12 12:39:42,376: ERROR/Worker-1] Error sending data
Traceback (most recent call last):
  File "/opt/mayan-edms/local/lib/python2.7/site-packages/gnupg.py", line 156, in _copy_data
    outstream.write(data)
IOError: [Errno 32] Broken pipe
[2018-11-12 12:39:42,688: ERROR/Worker-1] Error sending data
Traceback (most recent call last):
  File "/opt/mayan-edms/local/lib/python2.7/site-packages/gnupg.py", line 156, in _copy_data
    outstream.write(data)
IOError: [Errno 32] Broken pipe
I am kind of guessing that this may be an issue with permissions with the destination of where it is trying to write the scanned file to? If you can give me a pointer as to where it writes the scanned image I will check the permissions.

EDIT: I have done some further research... it is looking like this may not be a permissions error, as when scanning a single page, although I get the error above it always stores the first page successfully.

If I scan a document from the CLI by performing the following

Code: Select all

root@mayan-edms:~# scanimage -v --format=jpeg -p -d 'brother4:net1;dev0' > ~/brscan/test2.jpeg
scanimage: rounded value of br-x from 215.9 to 215.88
scanimage: rounded value of br-y from 355.6 to 355.567
scanimage: scanning image of size 1648x2799 pixels at 24 bits/pixel
scanimage: acquiring RGB frame
scanimage: min/max graylevel value = 0/255
scanimage: read 13838256 bytes in total
the scan is successful, but it ignores the next pages, and just empties the ADF

Whereas, if I do the following:

Code: Select all

root@mayan-edms:~# scanimage -b -v --format=jpeg -p -d 'brother4:net1;dev0' > ~/brscan/test3.jpeg
scanimage: rounded value of br-x from 215.9 to 215.88
scanimage: rounded value of br-y from 355.6 to 355.567
Scanning -1 pages, incrementing by 1, numbering from 1
Scanning page 1
scanimage: scanning image of size 1648x2799 pixels at 24 bits/pixel
scanimage: acquiring RGB frame
scanimage: min/max graylevel value = 0/255
scanimage: read 13838256 bytes in total
Scanned page 1. (scanner status = 5)
Scanning page 2
scanimage: scanning image of size 1648x2799 pixels at 24 bits/pixel
scanimage: acquiring RGB frame
scanimage: min/max graylevel value = 0/255
scanimage: read 13838256 bytes in total
Scanned page 2. (scanner status = 5)
Scanning page 3
scanimage: sane_start: Document feeder out of documents
This results in multiple pages being scanned, and multiple files being created:

Code: Select all

root@mayan-edms:~# ls
brscan	out1.jpg  out2.jpg
(ignore brscan, that is just a folder)
So the addition of -b on the scanimage command it activates batch scanning... this is when using an ADF.

@rosarior When documents are being scanned normally, do they create one long document, or are the documents created as separate pages and then compiled back into a single document for Mayan?

Further information:
Not sure if this is of any help, but this is the scanner-specific information that I have for my scanner:

Code: Select all

Options specific to device `brother4:net1;dev0':
  Mode:
    --mode Black & White|Gray[Error Diffusion]|True Gray|24bit Color|24bit Color[Fast] [24bit Color[Fast]]
        Select the scan mode
    --resolution 100|150|200|300|400|600|1200|2400|4800|9600dpi [200]
        Sets the resolution of the scanned image.
    --source FlatBed|Automatic Document Feeder(left aligned)|Automatic Document Feeder(centrally aligned) [Automatic Document Feeder(left aligned)]
        Selects the scan source (such as a document-feeder).
    --brightness -50..50% (in steps of 1) [inactive]
        Controls the brightness of the acquired image.
    --contrast -50..50% (in steps of 1) [inactive]
        Controls the contrast of the acquired image.
  Geometry:
    -l 0..215.9mm (in steps of 0.0999908) [0]
        Top-left x position of scan area.
    -t 0..355.6mm (in steps of 0.0999908) [0]
        Top-left y position of scan area.
    -x 0..215.9mm (in steps of 0.0999908) [215.88]
        Width of scan-area.
    -y 0..355.6mm (in steps of 0.0999908) [355.567]
        Height of scan-area.
It appears that the only way I can get the scanning to work is to leave a majority (apart from the resolution) of the drop-downs as 'scanner default'. I was wondering if there is a way that a 'custom' box could be added to enable specific switches?

Options in [ ] are default, so in effect, by default, it is scanning in 24bit Colour (fast), from the ADF (left aligned), with no contrast or brightness changes.

I've also got a suggestion that I am going to put in the suggestions box to request that the document source be changeable on the 'new document' page. The reason is that there are times when the scanned documents are single-sided, which I would like to use ADF for scanning... but when the document is double-sided I will need to use flatbed. I could just create a second source of scanner flatbed, but it is just a little bit clunky... :)

More information:
ok... so having a further dig around I found that the temp storage is /tmp in the container... so I thought I would have a look. When the document is being scanned I can see the first 'temp' page being created... then, as it gets to the second page, I can see the temp second page be created... but then the second page is ejected from the ADF, and I am left with this in /tmp

Code: Select all

-rw------- 1 mayan mayan 65524 Nov 12 14:18 brscan_jpeg_PAGE2_hOawhr
So it looks like it is trying to do the second page maybe?

Thanks for your help.

Kevin

Post Reply