Latency to copy files from hard disk to USB Flash disk

The intent of this forum is to discuss my DOS TSR programs (available at http://bretjohnson.us), how they work and don't work, new/missing features, status of updates, and anything else related to them that may need to be discussed.

Latency to copy files from hard disk to USB Flash disk

Postby arete » Wed May 18, 2016 7:27 am

Running Freedos 1.1 and the latest USB drivers (extracted from usbdos.zip) from Bret Johnson's website on a Dell Dimension T600r with 768 MB RAM.

Booted Freedos (using option 1 - using JEMMEX) normally off the hard drive.

At the command prompt, ran USBUHCI followed by USBDRIVE /T:Y /X:64

Observed that USBUHCI loads into high memory by default and USBDRIVE loads into low memory by default.

The USB drive I'm using for this is a 256MB drive formatted - FAT32 with 512 byte Allocation Unit Size.

When I attempt to copy a file of 1239 bytes from the hard drive - C: to the USB drive E:, it takes over a minute to copy.

Is this normal behavior or can this be made any faster?
arete
 
Posts: 4
Joined: Wed May 18, 2016 7:08 am

Re: Latency to copy files from hard disk to USB Flash disk

Postby Bret » Thu May 19, 2016 5:00 am

There are several factors that can affect the speed.

The first is the /X:# parameter with USBDRIVE. You have it set to 64, which is usually faster than the default of 1. But, you normally need to experiment with it to find the optimal value. As noted in the documentation, 64 can actually be slower than something like 4. I will hopefully have the speed issues resolved in the next version of USBDRIVE.

The second is the use of FAT32. The first time you read from (or write to, which also requires reading) a FAT32 disk, DOS must read the entire file Allocation Table (FAT). This can be a VERY long process -- well over a minute in some cases. the second time you read the disk, it is much faster. For a small disk (256 MB is a small disk), you're actually much better off using FAT16 instead of FAT32. Your cluster size may not be 512 bytes any more, but you'll get much better performance.

The third is the bus speed. USBUHCI only works at USB 1 speeds (max 12 Mbps). The way USB disks work, though, you can never even get close to 12 Mbps of real throughput. You can increase speed by moving up to USB 2, but my drivers don't currently do that -- you'll need to use another driver (or let your BIOS do it, if your BIOS supports USB disks). The next version of my drivers will support USB 2 speeds.
Bret
 
Posts: 478
Joined: Fri Oct 10, 2008 3:43 am
Location: Rio Rancho, NM

Re: Latency to copy files from hard disk to USB Flash disk

Postby arete » Thu May 19, 2016 5:25 am

Bret,

Appreciate the response. I'll experiment some more to see the optimal settings that would work for my drive including reformatting the drive to FAT16. The USB controller on this Dell motherboard does not support USB 2.0 or higher so I did interpret that I'm limited to USB 1.1 speeds. I'm hoping to reduce the latency that I'm experiencing but not holding my breath seeing that I do not have too many potential tweaks to work with :). Can't imagine a more ideal situation here where I'm trying to copy a single file of 1239 bytes to a blank USB drive.
arete
 
Posts: 4
Joined: Wed May 18, 2016 7:08 am

Re: Latency to copy files from hard disk to USB Flash disk

Postby Bret » Thu May 19, 2016 12:37 pm

You aren't necessarily limited to USB 1 speeds. If you have a spare PCI slot, you can plug in a USB 2 controller. But, the BIOS almost certainly won't work with it and you'll need some sort of external driver. Your computer is old enough that the BIOS most likely doesn't support USB disks. In the early development stages of USB all the BIOS supported (usually called "Legacy Support") was keyboards and/or mice (if they supported anything at all). USB disk support in the BIOS is commonplace these days.
Bret
 
Posts: 478
Joined: Fri Oct 10, 2008 3:43 am
Location: Rio Rancho, NM

Re: Latency to copy files from hard disk to USB Flash disk

Postby arete » Mon Mar 20, 2017 7:26 am

Continuing to explore USBUHCI and USBDRIVE with no success.

On my last test I used:

USBUHCI /DisableLegacy
USBDRIVE /X:4

The drive is a 128MB USB drive formatted as a FAT16 drive. Drives.com reports it as 133MB, FAT16.

It gets drive letter E: assigned to it.

When I run xcopy to get all the files on it, it fails consistently after copying the first two files.

The error msg:

"USB dos error reading from drive E: dos area: general failure"

Have tried giving no flags to USBDRIVE and with a /X:16 all resulting in the same failure.

Also tried another drive with the same results though this time it fails while attempting to copy the first file.
arete
 
Posts: 4
Joined: Wed May 18, 2016 7:08 am

Re: Latency to copy files from hard disk to USB Flash disk

Postby arete » Mon Mar 20, 2017 10:15 am

Finally success...or so I think...and I'm sure this won't apply to most of you because how specific your own implementation may be...but I'm writing this just in case it does.

Can't precisely say what fixed it for me, though I have a general idea.

So my task was to get USB working on a machine running FreeDOS so I could copy and transfer files to this machine without having to deal with either floppys nor CDs. The version (1.1) of FreeDOS that I installed was from a CD image, and it gave me the following boot options: (excerpt from my fdconfig.sys)

> MENU 1 - Load FreeDOS with JEMMEX, no EMS (most UMBs), max RAM free
> MENU 2 - Load FreeDOS with EMM386 (Expanded Memory) and SHARE loaded
> MENU 3 - Load FreeDOS including XMGR XMS-memory driver

123?DOS=HIGH
12?DOS=UMB
12?DOSDATA=UMB
1?DEVICE=C:\FDOS\BIN\JEMMEX.EXE NOEMS X=TEST I=TEST NOVME NOINVLPG
2?DEVICE=C:\FDOS\BIN\HIMEMX.EXE
2?DEVICE=C:\FDOS\BIN\JEMM386.EXE X=TEST I=TEST I=B000-B7FF NOVME NOINVLPG
3?DEVICE=C:\FDOS\BIN\XMGR.SYS
3?SHELL=C:\FDOS\bin\4dos.com C:\FDOS\bin /E:1024 /P:C:\FDOS\FDAUTO.BAT
12?SHELLHIGH=C:\FDOS\BIN\COMMAND.COM C:\FDOS\BIN /E:1024 /P=C:\FDOS\FDAUTO.BAT


If I pick menu option 1 or 2 above, and then run:

USBUHCI /DisableLegacy /DisableOther
USBDRIVE /X:4

It did not matter whether I had a /X flag or one with any value 2 or above.

Both drivers load successfully (can never seem to load them in upper memory even with the flag settings). If I now attempt to copy files from my USB drive using copy or xcopy, I consistently get the following failure.

"USB dos error reading from drive E: dos area: general failure"

If I select Menu 3 followed by running USBUHCI and USBDRIVE as above, the copy and xcopy succeeds without any issue even with a greater value in the /X flag as below:

USBUHCI /DisableLegacy /DisableOther
USBDRIVE /X:64

Further with menu option 3, any file that has a "Read Only" attribute set, fails to copy from the USB drive and I get the standard (Abort, Retry, Ignore, Fail) error msg.

So success or failure seems to be related to both choice of memory manager and shell as they are different based on the Menu options presented by my FreeDOS install.

The files seem to copy over consistently with Menu Option 3 which uses XMGR and 4DOS, though I'd like to understand the incompatibility with JEMMEX, HIMEMX, and JEMM386.

My PC is Dell based Pentium III 600MHz with 768MB of RAM.
arete
 
Posts: 4
Joined: Wed May 18, 2016 7:08 am

Re: Latency to copy files from hard disk to USB Flash disk

Postby Bret » Wed Mar 22, 2017 8:07 am

Arete:

Very interesting -- glad you finally got it to work. Next time I get a chance I'll experiment with the different memory managers to see if/how they make a difference for me. I don't think I ever experimented with different memory managers like that for USB compatibility. I do know I've had issues on different computers with different memory managers not working at all (e.g., JEMM would work when MS HIMEM would not, or vice versa). The only ones I really ever use and experiment with, at least on a regular basis, are JEMM and MS HIMEM -- I don't really ever use HIMEMX or XMGR.

I'm not sure why 4DOS would make a difference, or why read-only would keep you from copying a file. The USB drivers don't care about the read-only bit -- they just deliver the information that is stored on the USB drive and the OS decides whether or not it should do the copy.

I suspect the memory manager issues may have to do with the handling of the A20 line, which allows access to memory above 1 MB. The A20 line must be on for the USB drivers to work. There are actually several ways to toggle the A20 line, not all of them work on all computers, and some are faster than others. You should probably try and configure your memory manager to always keep the A20 line on, and may also want to try the different methods of toggling the A20 line (the memory manager should have options to control those things). If you don't use a memory manager at all, the USB drivers will try to toggle the A20 line themselves when they need to, which can slow things down though I don't think it should ever cause problems like you're seeing.
Bret
 
Posts: 478
Joined: Fri Oct 10, 2008 3:43 am
Location: Rio Rancho, NM


Return to Programs

Who is online

Users browsing this forum: No registered users and 3 guests

cron