How to enable the ath9k wireless driver on Ubuntu Lucid (10.04)

Ath9k is the latest wireless driver for the newer Atheros chipsets and just like the MadWifi driver could be used to put your NIC in a promiscuous mode, do packet injection, etc. in order to crack WEP.

Check the compatibility of your wireless NIC first.
To find out what kind of chipset you have, run:

lspci | grep -i wireless

The ath9k driver is present in any kernel starting with version 2.6.27. The oldest supported ath9k version comes with kernel 2.6.32. That means most likely you already have the driver and there is no need to download it, all you have to do is enable it.

The general steps in order to enable ath9k are:

  1. Get the kernel source and some necessary packages
  2. Configure the kernel
  3. Compile and install the kernel
  4. Configure grub to boot from the new kernel image

Some steps might sound a bit intimidating, but actually they are very easy to do.

Since you will be running a lot of commands as root, it would be more convenient to open a bash shell as root instead of preceding all commands with sudo. The easiest way to do this is to run the following command it terminal:

$ sudo bash

Note: The above command has nothing to do with enabling the root login. I strongly recommend to never enable the root login due to serious security risks! Some blogs might ask you to do that, but there is hardly ever any reason to do this.

1. Get the kernel source and some necessary packages

# aptitude install git-core build-essential kernel-package qt3-dev-tools libqt3-mt-dev fakeroot
# apt-get install linux-source 

The above command downloaded the kernel source tarball in /usr/src.
In my case the kernel version I am working with is 2.6.32 (patch level 15). Replace these numbers with the version of your kernel source.
Now lets cd to the place we downloaded the kernel source:

# cd /usr/src

Uncompress and extract the kernel source:

# tar xjvf linux-source-2.6.32.tar.bz2
# cd linux-source-2.6.32

2. Configure the kernel

Before configuring the kernel, lets import the configuration of the currently running kernel. This way we do not miss to enable something we currently have and need:

# cp -vi /boot/config-`uname -r` .config

Now we are ready to add a few configurations to the kernel:

# make xconfig

This opens up an xwindow with a hierarchical structure representing the kernel settings:

Linux Kernel Configuration
Linux Kernel Configuration

First enable mac80211:

Networking  --->
  Wireless  --->
     Improved wireless configuration API
     Generic IEEE 802.11 Networking Stack (mac80211)

You can then enable ath9k in the kernel configuration under:

Device Drivers  --->
  [*] Network device support  --->
        Wireless LAN  --->
             Atheros 802.11n wireless cards support

If you re-used the existing configuration, note that Ubuntu kernels build with debugging information on, which makes the resulting kernel modules (*.ko files) much larger than they would otherwise be. To turn this off, go into “Kernel hacking”; then, under “Kernel debugging”, turn OFF “Compile the kernel with debug info”.

Save changes and exit.

3. Compile and install the kernel

First, let’s ensure a “clean” build:

# make-kpkg clean

Since the kernel compilation might take some time, it is best to set the CONCURRENCY_LEVEL variable, so that it can take a full advantage of the multiple processors or mulitple CPU cores on the machine. The CONCURRENCY_LEVEL variable is equal to the number of processors on the machine plus one. So if your machine has a dual core processor the variable will be equal to 3. To find out how many processors or cores your computer has run:

# lshw -C CPU

In my case I get cpu:0 and cpu:1, which means that I have a dual core processor.
Besides the CONCURRENCY_LEVEL, the only other thing we need to pay attention to here is the “–append-to-version” option. It basically adds that string to the end of the kernel name and could be set to anything. That gives you the opportunity to put there something meaningful to distingush the kernel from the other ones. In this case I called it “-mykernel”.

Now you are ready to compile:

# CONCURRENCY_LEVEL=3 fakeroot make-kpkg --initrd --append-to-version=-mykernel kernel-image kernel-headers

If you want to see the ubuntu splash screen (or use text mode) before you get to X instead of just a black screen, you’ll want to make sure the framebuffer driver loads:

# echo vesafb >> /etc/initramfs-tools/modules
# echo fbcon >> /etc/initramfs-tools/modules

Now install the kernel and the headers from the created Debian packages:

# dpkg -i linux-image-2.6.32.15+drm33.5-mykernel_2.6.32.15+drm33.5-mykernel-10.00.Custom_i386.deb
# dpkg -i linux-headers-2.6.32.15+drm33.5-mykernel_2.6.32.15+drm33.5-mykernel-10.00.Custom_i386.deb

Now let’s generate an initramfs (initrd) image, that will be loaded up by grub on boot:

# cd /boot

To avoid confusion with the next command, here is what my /boot directory contains before I run the mkinitramfs command:

# ls -altr
total 27700
-rw-r--r--  1 root root  160280 2010-03-23 05:37 memtest86+.bin
-rw-r--r--  1 root root 4034976 2010-08-20 14:22 vmlinuz-2.6.32-24-generic
-rw-r--r--  1 root root 1689036 2010-08-20 14:22 System.map-2.6.32-24-generic
-rw-r--r--  1 root root  115905 2010-08-20 14:22 config-2.6.32-24-generic
-rw-r--r--  1 root root  651618 2010-08-20 14:22 abi-2.6.32-24-generic
-rw-r--r--  1 root root    1196 2010-08-20 14:24 vmcoreinfo-2.6.32-24-generic
drwxr-xr-x 22 root root    4096 2010-09-04 15:27 ..
-rw-r--r--  1 root root       0 2010-09-07 22:11 initrd.img-2.6.31-wl
-rw-r--r--  1 root root 7977082 2010-09-07 22:49 initrd.img-2.6.32-24-generic
-rw-r--r--  1 root root  115845 2010-09-08 02:13 config-2.6.32.15+drm33.5-mykernel
-rw-r--r--  1 root root 3999488 2010-09-08 07:38 vmlinuz-2.6.32.15+drm33.5-mykernel
-rw-r--r--  1 root root 1644782 2010-09-08 07:38 System.map-2.6.32.15+drm33.5-mykernel
drwxr-xr-x  3 root root    4096 2010-09-08 16:01 .

Run the mkinitramfs to generate the initrd:

# mkinitramfs -k -o initrd.img-2.6.32.15+drm33.5-mykernel 2.6.32.15+drm33.5-mykernel

4. Configure grub to boot from the new kernel image

# update-grub2

The last thing to do is to ensure that the ath9k module loads on boot. That will avoid running modprobe ath9k after each reboot.
So lets add the line ath9k to the end of the /etc/modules file:

# echo ath9k >> /etc/modules

Now reboot your computer and chose the new kernel to boot into from the grub menu.

Your wireless NIC should be working now using the ath9k driver.


Credits:

Android – Displaying Dialogs From Background Threads

Having threads to do some heavy lifting and long processing in the background is pretty standard stuff. Very often you would want to notify or prompt the user after the background task has finished by displaying a Dialog.

The displaying of the Dialog has to happen on the UI thread, so you would do that either in the Handler object for the thread or in the onPostExecute method of an AsyncTask (which is a thread as well, just an easier way of implementing it). That is a textbook way of doing this and you would think that pretty much nothing wrong could go with this.

Surprisingly I found out that something CAN actually go wrong with this. After Google updated the Android Market and started giving crash reports to the developers I received the following exception:

android.view.WindowManager$BadTokenException: Unable to add window — token android.os.BinderProxy@447a6748 is not valid; is your activity running?
at android.view.ViewRoot.setView(ViewRoot.java:468)
at android.view.WindowManagerImpl.addView(WindowManagerImpl.java:177)
at android.view.WindowManagerImpl.addView(WindowManagerImpl.java:91)
at android.view.Window$LocalWindowManager.addView(Window.java:424)
at android.app.Dialog.show(Dialog.java:239)
at android.app.Activity.showDialog(Activity.java:2488)

at android.os.Handler.dispatchMessage(Handler.java:99)

I only got a couple of these exceptions from thousands of installs, so I knew that was not anything that happens regularly or that it was easy to replicate.

Looking at the stack trace above it gives us a pretty good idea why it failed. It started in the Handler object, which naturally was called by a background thread after it finished its processing. The Handler instance tried to show a Dialog and before it could show it, it tried to set the View for it and then it failed with:

android.view.WindowManager$BadTokenException: Unable to add window — token android.os.BinderProxy@447a6748 is not valid; is your activity running?

The 447a6748 number is just a memory address of an object that no longer exists.

Note- do not get hung up on the exact number. It would be different with every execution.

Now we know why the application crashed, the only thing left is to figure out what caused it?

We know that background threads execute independently of the main UI thread. That means that the user could be interacting with the application during the time that the thread is doing its work under the covers. Well, what happens if the user hits the “Back” button on the device while the background thread is running and what happens to the Dialog that this thread is supposed to show? Well, if the timing is right the application will most likely crash with the above described error.

In other words what happens is that the Activity will be going through its destruction when the background thread finishes its work and tries to show a Dialog.

In this case it is almost certain that this should have been handled by the Virtual Machine. It should have recognized the fact that the Activity is in the process of finishing and not even attempted to show the Dialog. This is an oversight of the Google developers and it will probably be fixed some time in the future, but in the meantime the burden is on us to take care of this.

The fix to this is pretty simple. Just test if the Activity is going through its finishing phase before displaying the Dialog:

private Handler myHandler = new Handler() {
  @Override
  public void handleMessage(Message msg) {
    switch (msg.what) {
      case DISPLAY_DLG:
        if (!isFinishing()) {
        showDialog(MY_DIALOG);
        }
      break;
    }
  }
};

Android applications that use the MyLocationOverlay class crash on the new Droid X


Update: August 29, 2010
Tested this myself on a friend’s phone and this bug is definitively not fixed yet despite the recent Motorola updates! The burden is on you (the developer) to fix this if you want your applications to run on a Droid X.
Update: August 23, 2010
Still getting reports that the update issued by Motorola last month did not fix this issue!


As soon as the new Motorola Droid X came out I started getting reports that my applications would crash on it.

The description I was getting from people was that the application would start, then it would show the Google map and start moving the map and zooming in on to their current location and then all of a sudden the app would crash. By crash I mean they get taken back to the phone’s home screen. No “Force Close” dialog pops up… nothing… just gets back to the home screen.

If both the GPS and the Wireless Networks are turned off from the Locations settings then the applications would work fine, but of course that means that the current location of the device would be impossible to find.

At this point I knew that the problem had to do with either the LocationManagers or the LocationOverlays but I had no way of finding out exactly what the problem was until I could get my hands on a Droid X phone and connect it to my computer to look at the execution stack.

Well it turns out the problem is with the default MyLocationOverlay class. The Droid X phones throw an exception when they try to draw the dot showing the location of the device:

E/AndroidRuntime(10458): Uncaught handler: thread main exiting due to  uncaught exception
E/AndroidRuntime(10458): java.lang.ClassCastException:  android.graphics.drawable.BitmapDrawable
E/AndroidRuntime(10458): 	at com.google.android.maps.MyLocationOverlay.getLocationDot(MyLocationOverlay.java:180)
E/AndroidRuntime(10458): 	at com.google.android.maps.MyLocationOverlay.drawMyLocation(MyLocationOverlay.java:561)

Another developer (rgfindl) who had the same problem like I did and who had already made the trip to the Verizon store gave me the above stack trace.

Several possible solutions popped into my head:

1. Stop using the MyLocationOverlay class and start relying only on the LocationManager classes and draw/redraw my current location onLocationChanged.
2. Extend from MyLocationOverlay class and override the draw method.
3. Stop supporting the Droid X.

None of these solutions are ideal with the last one being almost unacceptable.

Luckily the same developer also gave me a link to a solution implementing the 2nd option above, so I did not even have to code it myself. Apparently that is not the first time Motorola phones (Motorola Cliq and Motorola Dext) have a problem with this class. Most likely their builds are missing the drawable resource.

Anyway, the solution consists of the .java file that implements the FixedMyLocationOverlay class, which inherits from the default MyLocationOverlay class and overrides the drawMyLocation method. And a .png file with a dot that will be representing the current location on the map.

Just use this derived class FixedMyLocationOverlay instead of the default MyLocationOverlay. If you look at the implementation of the drawMyLocation method, you will see that if the phone has no problem it will use the default implementation of the parent class, but if it throws an exception it will use the custom code to draw the location.

I would think that there are a lot of applications in the Android Market right now that use the default MyLocationOverlay class and all of them will be crashing on the new Droid X until either Motorola gets their act together or the developers realize the issue and work around it.

Just Upgraded my Nexus One from Android 2.1 to Android 2.2

The latest Android OS version 2.2 (“Froyo”) was announced at the Google I/O conference on the 11th of May. As you already probably know it adds multiple enhancements like tethering, wifi hotspot functionality, full support for Flash, etc. Since I use my Nexus One phone for Android development, I did not want to download the update from Google and then install it with adb. I just wanted to wait for the over the air update.

And to my surprise, the first time I looked at my phone this morning, I had this message on the screen:

Android 2.2 Upgrade Message
Android 2.2 Upgrade Message

I was not sure how long it would take, so I just hit “Install Later”. Then a couple of hours later, when I was ready, I just went to Settings -> About Phone -> Software Update and hit “Restart & Install”:

Android 2.2 Settings -> About Phone -> System Updates

The phone rebooted and started the upgrade. The whole process with the installation took less than 5 minutes.

I am not going to go into details what the features of Android 2.2 are. There are so many announcements and blog posts on this matter already out there…

But what I am going to say is that Android is the leading OS in the mobile phone market right now. I can say this definitively, since I have been using iPhones for the past 2 years. I have a 3G and a 3GS phones. I encourage any iPhone user to go and check out an Android device. Please do not look at the low end hardware that runs Android (like the Cliq, the old G1, etc.). Check out the HTC Evo, the HTC Incredible, the Nexus One etc. See for yourselves before you make your decision to get another phone and lock into a 2 year contract. I will mention only a few things you will discover on the Android device that your phone probably does not have right now:

  1. Turn by turn voice navigation
  2. Built in tethering
  3. Built in WiFi hotspot capability supporting multiple devices
  4. Full support for Flash
  5. Full blown multitasking on the OS level (no need for the programmer to do anything special)
  6. Integrated speech recognition across the board. Voice input could be used anywhere in place of the text input.
  7. Over the air updates and syncing. That means no iTunes or the like!

And many other features that I do not want to get into details right now. I am sure you can fill in the gaps for yourselves…

Find the Geopgraphical Location of an IP Address

There are a few applications that give the geographical location of an IP address *. My favorite one is VisualRoute. It not only maps the location of the source and destination IP addresses, but also all the hops in between. The down side to it is that it is not available for Linux and it costs $50 per user.

The http://mapulator.com site gives you pretty much the same information online for free, but it has been down for quite some time now.

Lately I have been using http://whatismyipaddress.com/ip-lookup. But it only maps the destination IP address.

If anyone knows of a Linux application that would map all the hops and the final IP destination, please leave a comment!

* IP addresses do not have a geographical location per se. Any program that maps IP addresses can only give you where the whole class of IP addresses (that the specified IP address belongs to) is registered to be located at. They do this by querying a database that has that information. This is accurate down to a city level at best. Do not expect to find where someone lives by their IP address!

Install Subversion and Subclipse for Eclipse on Ubuntu

If you do not have Eclipse installed yet, run:

sudo apt-get install eclipse

Install Subversion

sudo apt-get install subversion

Install the Subversion plugin for Eclipse

Get the Java bindings for Subversion:

sudo apt-get install libsvn-java

Now we are ready to install the plugins in Eclipse.

Open Eclipse.

Go to Help -> Install New Software

Then hit the “Add” button.

Put “Subclipse 1.6.x (Eclipse 3.2+)” under Name and “http://subclipse.tigris.org/update_1.6.x” under Location:

Add the libraries for Subclipse
Add the libraries for Subclipse

The above is for Elclips 3.2+ and Subversion 1.6.x.

If you have different Eclipse or Subversion versions, check this list and substitute accordingly:

Name: Subclipse 1.6.x (Eclipse 3.2+)
URL: http://subclipse.tigris.org/update_1.6.x

Name: Subclipse 1.4.x (Eclipse 3.2+)
URL: http://subclipse.tigris.org/update_1.4.x

Name: Subclipse 1.2.x (Eclipse 3.2+)
URL: http://subclipse.tigris.org/update_1.2.x

Name: Subclipse 1.0.x (Eclipse 3.0/3.1)
URL: http://subclipse.tigris.org/update_1.0.x

More info about the different versions: http://subclipse.tigris.org/servlets/ProjectProcess?pageID=p4wYuA

Then hit “OK”. You will now be presented with these options:

Subclipse Installation Options
Subclipse Installation Options

If you are unsure what Subclipse component you will need, you can check all of them. Also, keep in mind that you can always go back to this in the future and install/uninstall any of the components as needed.

Then “Next” and “Finish” on the subsequent screens.

After the plugins are installed, it will prompt you to restart Eclipse. Go ahead and do so.

You are not done yet. You need to fix the JavaHL.

Edit the eclipse.ini file:

sudo vi /usr/lib/eclipse/eclipse.ini

Add the following line under -vmargs:

-Djava.library.path=/usr/lib/jni

Edit (2012-11-23): The above path is for a 32bit installation. The 64bit installation’s library path is /usr/lib/x86_64-linux-gnu/jni. If you are unsure the JavaHL library is on your machin, just search for libsvnjavahl-1.so:

sudo find / -name "libsvnjavahl-1.so"

That will give you the path you need to add to the eclipse.ini file.

Here is what my eclipse.ini file looks like:

$ cat /usr/lib/eclipse/eclipse.ini
-startup
plugins/org.eclipse.equinox.launcher_1.0.201.R35x_v20090715.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.gtk.linux.x86_1.0.200.v20090520
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
-startup
--launcher.library
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
-startup
/usr/lib/eclipse/plugins/org.eclipse.equinox.launcher_1.0.201.R35x_v20090715.jar
--launcher.library
/usr/lib/eclipse/plugins/org.eclipse.equinox.launcher.gtk.linux.x86_1.0.200.v20090520
-vmargs
-Djava.library.path=/usr/lib/jni
-Dorg.eclipse.equinox.p2.reconciler.dropins.directory=//usr/share/eclipse/dropins
-Xms40m
-Xmx256m
-Dorg.eclipse.equinox.p2.reconciler.dropins.directory=//usr/share/eclipse/dropins

For more info on fixing JavaHL if you need it: http://subclipse.tigris.org/wiki/JavaHL#head-bb1dd50f9ec2f0d8c32246430c00e237d27a04fe

You should be able to use Subversion with Eclipse for your projects at this point.

How to connect your Android phone to Ubuntu to do development, testing, installations or tethering

If you do any Android development in Ubuntu it is much better to test your applications on a real device rather than just relying on the emulator.

If you just connect the phone to the computer through USB and run adb devices you will find out that your phone is not recognized:

$ adb devices
List of devices attached
???????????? no permissions

This is easily fixable in Ubuntu. You have to add a line to a file in the /etc/udev/rules.d directory.

What line in which file depends on the manufacturer and model of the device. You can find out the Vendor ID number and Product ID number of the device by running (after you connected it via USB of course):

$ lsusb

For example, if you had a Nexus One connected you would get:

Bus 002 Device 004: ID 18d1:4e12

In this case the Vendor Id is “18d1” and the Product ID is “4e12”. Please keep in mind that the Vendor ID for HTC changed from “0bb4” to “18d1”. The older HTC phones like the G1 have a Vendor ID of “0bb4”.

Here is how to set up some of the major Android phones:

Step 1. Enable Unknown sources and USB debugging:

Go to Settings -> Applications and check the Unknown sources check-box, then go to Development and enable the USB debugging.

Step 2. Depending on your phone follow these instructions:

  • G1
    Create/edit a file in /etc/udev/rules.d called 51-android.rules:

    $ sudo gedit /etc/udev/rules.d/51-android.rules

    Add the following line to it and save it:

    SUBSYSTEMS==”usb”, ATTRS{idVendor}==”0bb4″, ATTRS{idProduct}==”0c01″, MODE=”0666″

  • HTC Hero

    Add to the file (or create it if it does not exist- see above) /etc/udev/rules.d/51-android.rules the following line:

    SUBSYSTEMS==”usb”, ATTRS{idVendor}==”0bb4″, ATTRS{idProduct}==”0c02″, MODE=”0666″

  • HTC EVO

    Add to the file (or create it if it does not exist- see above) /etc/udev/rules.d/51-android.rules the following line:

    SUBSYSTEMS==”usb”, ATTRS{idVendor}==”0bb4″, ATTRS{idProduct}==”0c8d”, MODE=”0666″

  • HTC EVO Shift

    Add to the file (or create it if it does not exist- see above) /etc/udev/rules.d/51-android.rules the following line:

    SUBSYSTEMS==”usb”, ATTRS{idVendor}==”0bb4″, ATTRS{idProduct}==”0ca5″, MODE=”0666″

  • HTC ThunderBolt

    Add to the file (or create it if it does not exist- see above) /etc/udev/rules.d/51-android.rules the following line:

    SUBSYSTEMS==”usb”, ATTRS{idVendor}==”0bb4″, ATTRS{idProduct}==”0ca4″, MODE=”0666″

  • LG OPTIMUS LG-P506

    Add to the file (or create it if it does not exist- see above) /etc/udev/rules.d/51-android.rules the following line:

    SUBSYSTEMS==”usb”, ATTRS{idVendor}==”1004″, ATTRS{idProduct}==”618e”, MODE=”0666″

  • Motorola Xoom

    Add to the file (or create it if it does not exist- see above) /etc/udev/rules.d/51-android.rules the following line:

    SUBSYSTEMS==”usb”, ATTRS{idVendor}==”22b8″, ATTRS{idProduct}==”70a9″, MODE=”0666″

  • Motorola Droid
    Create/edit a file in /etc/udev/rules.d called 10-motorola-droid.rules:

    $ sudo gedit /etc/udev/rules.d/10-motorola-droid.rules

    Add the following line to it and save it:

    SUBSYSTEMS==”usb”, ATTRS{idVendor}==”22b8″, ATTRS{idProduct} ==”41db”, MODE=”0600″

  • Nexus One
    Create/edit a file in /etc/udev/rules.d called 99-android.rules:

    $ sudo gedit /etc/udev/rules.d/99-android.rules

    Add the following line to it and save it:

    SUBSYSTEM==”usb”, ATTRS{idVendor}==”18d1″, SYMLINK+=”android_adb”, MODE=”0666″

Step 3. Restart udev:

$ sudo restart udev

Check if the device is recognized:

$ adb devices

You may need to stop and start the adb server:

$ adb kill-server
$ adb start-server

Update: Please note that some devices require to disconnect the usb cable at this point and then reconnecting it.

You should be good to go…

Extract Audio (.mp3) from Video Files Like .flv, .mov, .avi and Others with Ubuntu

It is very easy to extract the audio track from video files using Linux. All you need is ffmpeg and some codecs.

Let’s get started…

Note: The commands below are for Ubuntu (or Debian derivatives) but you can do the same with any other Linux distribution provided you can install the necessary packages.

1. Add the Medibuntu’s repository to your sources.list:

sudo wget --output-document=/etc/apt/sources.list.d/medibuntu.list http://www.medibuntu.org/sources.list.d/$(lsb_release -cs).list && sudo apt-get --quiet update && sudo apt-get --yes --quiet --allow-unauthenticated install medibuntu-keyring && sudo apt-get --quiet update

2. Install ffmpeg:

sudo apt-get install ffmpeg

3. Let’s get the restricted packages and some codecs installed:

sudo apt-get install ubuntu-restricted-extras libmp3lame0 libdvdcss2 w32codecs

The above command is for i386 architecture. If you have an amd64 architecture, substitute w32codecs with w64codecs.

Note: This is not a complete list of codec packages by any stretch of the imagination. It will get you started though and you will be able to do most formats, but you might have to add codecs as you go along.

4. Now we are ready to extract the audio from the video files:

ffmpeg -i input_file.flv output_file.mp3

The above command will extract the audio from a Flash video file. You can do the same for a QuickTime file as well:

ffmpeg -i input_file.mov output_file.mp3

Or for an Audio Video Interface file:

ffmpeg -i input_file.avi output_file.mp3

YouTube and other video web sites:

Having done all this, now we can download flash files from places like YouTube and strip the audio from them. All you need besides the steps above is a way to save the Flash files (.flv) from YouTube. An easy way to do that is by using Firefox Add-ons like Download Flash and Video or Flash Video Downloader.

The quality of the audio in the YouTube videos for example is 64 bit/sec. Most of the mp3 files are normally compressed to 128 bit/sec or above. Obviously the quality will not be the same, but a human year cannot tell the difference.

The quality of the extracted mp3 will depend on the quality of the audio track in the video file. So the above statement about the 64 bit/sec audio is mostly the case for the files on some video sharing sites.

How to Resume Partial File Transfers

I work primarily with UNIX and Linux machines and scp is my main choice to transfer files with. It is both convenient, short and secure.

Example:

scp localfile user@remotecomputer:/path/to/target/dir

Recently I was transferring an 8GB file and due to a network issue, the transfer was interrupted at nearly 40%.

I have learned over the years that there is often little that can be done to prevent such interruptions. Of course, this can be both frustrating and time-consuming, but there is a quick fix. Although such disruptions can’t be prevented, there is a fast and easy way to resume them. Resuming has relatively few system requirements, and can save a lot of time and hassle when transferring large files.

I found a solution at joen.dk ,which uses rsync to resume the transfer:

rsync --partial --progress --rsh=ssh local_file host:remote_file

NOTE: The syntax is ssh source_file target_file. So in the above example I am transferring a local file to a remote server. A few people did not pay close attention and switched the source and destination files and resulted in overwriting the original.

Now we can improve this slightly by shortening the above command. We can substitute –rsh=ssh with -e ssh, and use -P instead of –partial –progress. Also, you can add user@host if you need to specify a different remote shell user:

rsync -P -e ssh local_file user@host:remote_file

This above example will work with any file that was partially transferred. How the transfer was started does not really matter. It could be through scp, nc or even ftp. After you execute the above command it will take rsync a little time to verify the previously downloaded part before it continues with the rest. Be patient, depending on your network speed rsync could take some time to go through what you have already transferred. Of course this is much faster than if you were to start the download all over again and it shows you the progress in percentages.

Keep in mind that there have to be a couple of requirements in place in order to resume the file transfer with rsync:

1. You should have remote shell access.
2. The remote machine should have rsync installed. Since rsync is by default on most Linux distributions that generally should not be an issue.