Downloading files with Bash

It is a rare occasion when all the files needed are available locally and ready to go. This section goes through how to download files from a few different locations

curl

curl is a command used to download files from various places on the internet.

The simplest usage of curl with no modifiers will print the contents of the downloaded file to the terminal.

# Download some data from the internet and print to the terminal
curl 'https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data'

We can save this data to disk in a few different ways

# save the data into a filename that we specify
curl -o 'a_filename_of_my_choosing.csv' 'https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data'

# save the data with the same name as the original
curl -O 'https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data'

# redirect the output from simple curl command into a file
curl 'https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data' > 'save_output.csv'

Download data from a website that is password protected

curl -u <username>:<password> <url>

Download data from an FTP server using curl

curl -u user:password 'ftp://mysite/myfolder/myfile/raw.csv' -o ~/Downloads/raw.csv