Download All Documents from Blackboard’s Digital Dropbox

Blackboard sucks. This is the opinion shared by roughly 95% of faculty at my university. But this is what we have, and a crappy course management system is still better than no course management system. So we are stuck with it for the time being.

Most of the gripes about BB system concentrates around the Digital Dropbox feature which is both very convenient, and very annoying. It’s convenient for the students because they can easily submit their assignments from anywhere at any time. They simply log into blackboard, click on the appropriate course link, and upload the file. They don’t have to spend time printing their work, they do not need to search for instructors email, and hope the spam filter won’t snag it. They just hit a button and they are done.

It is horribly annoying because the instructor does not have any control over how the uploads are organized. When you open the instructors view of the dropbox you see a long list of download links in reverse chronological order (newest entries are on the top). There is no way to tag or categorize assignments so unless students name their files appropriately you have to open each file to figure out what it is.

But most annoying feature is probably the fact that there is no way to download all the files from the dropbox at once. You can only grab them one at a time by clicking the links. You’d think there was a button that says “download all” but there is none. I think there is silly trick some people use that deals with exporting/archiving the course but it’s not exactly what you’d want to do every week unless you have to. So I decided to create a solution that would let me easily download all the files from the dropbox at a click of a button. How? Observe:

use strict;
use WWW::Mechanize;
use MIME::Base64;


my $bb_login_url = 'http://school.blackboard.com/webapps/login/?action=relogin';
my $bb_dropbox_url = 'http://school.blackboard.com/bin/common/dropbox.pl?action=LIST&course _id=_SOME_NUMBER_1&render_type=EDITABLE';

my $bb_user = 'your_username';
my $bb_passwd = 'your_password';

# where do you want the downloaded files
my $folder = "absolute_path";

# log into bb
my $browser = WWW::Mechanize->new(autocheck => 1, quiet => 0);
$browser->get($bb_login_url);
$browser->form_number(1);
$browser->field('user_id', $bb_user);
$browser->field('encoded_pw', encode_base64($bb_passwd));
$browser->click();

# go to dropbox
$browser->get($bb_dropbox_url);

# grab all the links
my @l = $browser->links;

my $i = 1;

foreach (@l)
{
	# the links to files have the word uploads in the uri
	if($_->url() =~ m/uploads/)
	{	
		print $i . "--" . substr($_->url(), rindex($_->url(), '/')+1) . "\n";
		$browser->get($_->url(), ":content_file" => $folder . $i . "--" . substr($_->url(), rindex($_->url(), '/')+1));
		$i++;
	}
}

I could do this in less than 10 lines probably but we are not playing perl-golf here. You will need to hard code the appropriate URL’s. I hope you can figure out how to get them. If not, you should not be fucking around with perl. In fact, step away from the computer right now. These things can explode if you press the wrong key combination!

I guess the important bit of information here is the regex inside of the loop. The whole thing works because the URL’s for the files inside the dropbox (at least in the version of bb we use) look like this:

http://school.blackboard.com/courses/1/COURSE_ID/uploads/_SOMENUMBER_1  /somefile.doc

None of the other links on the page have the word uploads anywhere in the URL so this is how we can isolate the download links regardless of what type of files they are. The get() method of WWW::Mechanize can fetch the specified URI into a file instead of memory if you pass in “:content_file” => “desired_file_name_and_path” as the second argument.

I’m appending a number to the output file name because it is not uncommon to find 2 files with the same name (for example homework1.zip or something like that). The get method will overwrite files without warning so the sequential numbers prevent that from happening. They also help to organize files to match them up with the blackboard page. If your students do not include their name inside of the file, you can look back on the dropbox page and figure out which file was it rather easily.

To delete all the files from dropbox simply change the inner loop to this:

foreach (@l)
{
	if($_->url() =~ m/=REMOVE/)
	{	
		$browser->get(substr($_->url(), rindex($_->url(), '/')+1));
	}
}

Here we are grabbing the links that have the word REMOVE in the link. So now you can fetch the whole dropbox and delete everything from it for a clean start.

I’m not sure if anyone will find this useful, but it really makes working with BB a tiny little bit less annoying for me.

[tags]blackboard, digital dropbox, perl, automation[/tags]