Download All Documents from Blackboard’s Digital Dropbox
Blackboard sucks. This is the opinion shared by roughly 95% of faculty at my university. But this is what we have, and a crappy course management system is still better than no course management system. So we are stuck with it for the time being.
Most of the gripes about BB system concentrates around the Digital Dropbox feature which is both very convenient, and very annoying. It’s convenient for the students because they can easily submit their assignments from anywhere at any time. They simply log into blackboard, click on the appropriate course link, and upload the file. They don’t have to spend time printing their work, they do not need to search for instructors email, and hope the spam filter won’t snag it. They just hit a button and they are done.
It is horribly annoying because the instructor does not have any control over how the uploads are organized. When you open the instructors view of the dropbox you see a long list of download links in reverse chronological order (newest entries are on the top). There is no way to tag or categorize assignments so unless students name their files appropriately you have to open each file to figure out what it is.
But most annoying feature is probably the fact that there is no way to download all the files from the dropbox at once. You can only grab them one at a time by clicking the links. You’d think there was a button that says “download all” but there is none. I think there is silly trick some people use that deals with exporting/archiving the course but it’s not exactly what you’d want to do every week unless you have to. So I decided to create a solution that would let me easily download all the files from the dropbox at a click of a button. How? Observe:
use strict; use WWW::Mechanize; use MIME::Base64; my $bb_login_url = 'http://school.blackboard.com/webapps/login/?action=relogin'; my $bb_dropbox_url = 'http://school.blackboard.com/bin/common/dropbox.pl?action=LIST&course_id=_SOME_NUMBER_1&render_type=EDITABLE'; my $bb_user = 'your_username'; my $bb_passwd = 'your_password'; # where do you want the downloaded files my $folder = "absolute_path"; # log into bb my $browser = WWW::Mechanize->new(autocheck => 1, quiet => 0); $browser->get($bb_login_url); $browser->form_number(1); $browser->field('user_id', $bb_user); $browser->field('encoded_pw', encode_base64($bb_passwd)); $browser->click(); # go to dropbox $browser->get($bb_dropbox_url); # grab all the links my @l = $browser->links; my $i = 1; foreach (@l) { # the links to files have the word uploads in the uri if($_->url() =~ m/uploads/) { print $i . "--" . substr($_->url(), rindex($_->url(), '/')+1) . "\n"; $browser->get($_->url(), ":content_file" => $folder . $i . "--" . substr($_->url(), rindex($_->url(), '/')+1)); $i++; } }
I could do this in less than 10 lines probably but we are not playing perl-golf here. You will need to hard code the appropriate URL’s. I hope you can figure out how to get them. If not, you should not be fucking around with perl. In fact, step away from the computer right now. These things can explode if you press the wrong key combination!
I guess the important bit of information here is the regex inside of the loop. The whole thing works because the URL’s for the files inside the dropbox (at least in the version of bb we use) look like this:
http://school.blackboard.com/courses/1/COURSE_ID/uploads/_SOMENUMBER_1 /somefile.doc
None of the other links on the page have the word uploads anywhere in the URL so this is how we can isolate the download links regardless of what type of files they are. The get() method of WWW::Mechanize can fetch the specified URI into a file instead of memory if you pass in “:content_file” => “desired_file_name_and_path” as the second argument.
I’m appending a number to the output file name because it is not uncommon to find 2 files with the same name (for example homework1.zip or something like that). The get method will overwrite files without warning so the sequential numbers prevent that from happening. They also help to organize files to match them up with the blackboard page. If your students do not include their name inside of the file, you can look back on the dropbox page and figure out which file was it rather easily.
To delete all the files from dropbox simply change the inner loop to this:
foreach (@l) { if($_->url() =~ m/=REMOVE/) { $browser->get(substr($_->url(), rindex($_->url(), '/')+1)); } }
Here we are grabbing the links that have the word REMOVE in the link. So now you can fetch the whole dropbox and delete everything from it for a clean start.
I’m not sure if anyone will find this useful, but it really makes working with BB a tiny little bit less annoying for me.
Related Posts:
February 4th, 2008 at 2:46 pm (7967) [Quote]
Perhaps you should look into this Firefox extension which and I believe does what you need more conveniently:
http://www.downthemall.net/
I strongly recommend the beta 1.0b2 — it works great.
Posted usingFebruary 4th, 2008 at 3:04 pm (7968) [Quote]
Yeah, I used downthemall before but I forgot about it. Thanks for spoiling my fun here. ;P
Actually a small modification to my script will allow me to delete the files from BB dropbox after I downloaded them which helps to keep that damn thing clean and managable. I don’t think downthemall can do this.
Posted usingFebruary 5th, 2008 at 5:25 am (7975) [Quote]
Blackboard is indeed pretty outdated and flawed. I still don’t know how they managed to become a sort of standard within course management system/ e-learning system? I remember TWT (from the ESB of Porto) was pretty ok, but I don’t know if they still develop it. I think the guys at Laval University in Montréal also designed something very nice.
Here is another one which is pretty good: http://www.threeships.com/ ; Too bad they don’t have an English version of the website though.
Posted usingFebruary 5th, 2008 at 11:59 am (7977) [Quote]
I agree with all of the assessments made above. Sadly this is another example of being “first” to market and acquiring your biggest competitor (WebCT) which was a much better product.
It looks like we might be stuck to it for a while. I mean look at the URL: montclair.blackboard.com. WTF! Why is it hosted by them (I know why sadly and that might change soon) and even if it was hosted internally why do we have their brand name in the URL…why not just something like cms.montclair.edu (CMS = Course Management System) so that we can change our vendor without impacting user community.
Posted usingFebruary 5th, 2008 at 9:29 pm (7981) [Quote]
We used to host it, didn’t we? I remember that at one point the url was blackboard.montclair.edu. That was long time ago when I was a sophomore (or something like that). I do not remember using blackboard my freshman year.
Btw, I really think you guys should register cms.montclair.edu and point it at montclair.blackboard.com. In fact I’m surprised it hasn’t been done yet.
Btw - out of curiosity: how much storage space do we actually have with blackboard? I’m pretty sure it would be much cheaper to set up a local server with twice the amount of storage including backup and maintenance costs. It would be pile of money up front, but it would pay for itself in few semesters probably. And, you know - put it behind SSL for example. But then again, what do I know.
Posted usingFebruary 6th, 2008 at 10:59 am (7988) [Quote]
Luke Maciak said:
…and hopefully we will again very soon.
Luke Maciak said:
you and I both. Btw, it’s an OIT “controlled” service.
Luke Maciak said:
As far as I know, it’s incremental, not fixed, so it increases as we need it as long as it is reasonable.
Posted usingMay 14th, 2008 at 6:01 pm (9053) [Quote]
Is there any problem with deleting the subdirectories under the drop box? Are there any references to these links in the database that will be hosed once the directory and file are deleted?
Posted usingMay 26th, 2008 at 9:33 pm (9135) [Quote]
Take a look at moving over to Moodle or Sakai. Both Open Source and free. Will save your university a ton of money. Lots of 3rd party integrations. I have been using Moodle for years. check it out.
Posted usingMay 27th, 2008 at 9:43 am (9141) [Quote]
Andy said:
I’m sure we would. But rolling out a new service on my own, while the university as a whole is committed to blackboard and students already know how to use it, is probably counterproductive.
Posted usingAugust 11th, 2008 at 7:41 pm (9831) [Quote]
Neat perl trick, and certainly suitable to perl ninjas, but for techno-weenies like myself there’s a much easier solution: Downthemall!, an easy to use plugin for Firefox.
Install it, navigate to blackboard, right click. Easy as pie.
Posted using