As the doctor gone rogue

February 20, 2014

Download files from a webpage with wget.

Filed under: bash — Tags: , , — hypotheses @ 9:15 pm

Recently phylotree just release another update of mitochndrial phylogentic trees (19Feb2014). Besides the updated tree, a very nice feature of http://www.phylotree.org website is that they have a great curation of mitochondrial sequences publicly available for download. You can check out their website here http://www.phylotree.org/mtDNA_seqs.htm

If you have a curious mind, you may want to download all the sequences, construct your own trees or use the data to do something else creatively.

In this case wget might be your best friends, although you might be able to write a python script to do something similar.

 wget -r --accept "*.ext" --level 2 http://www.website.com/pagewithLink.html
 

As a reminder, if you are behind a proxy firewall, take a look at my previous post  https://bhoom.wordpress.com/2013/07/26/how-to-wget-with-proxy-authentication/

 

Advertisements

July 14, 2010

Using wget to download files from secured website

Filed under: bash — Tags: , — hypotheses @ 4:56 pm

Our collaborator just uploaded a bunch of files to their website today. One way to get those data easily is to download them all using wget, which mirror all the structures on the remote websites.

Normally you can


wget -r http://fly.srk.fer.hr --user=bhoom --password=bhoom_Password

-r : for recursive

You can also limit how many levels you want to download, the space limit, etc.

Create a free website or blog at WordPress.com.