Skip to content

Setup data tools

While the bash shell comes prepackaged with many useful tools for manipulating data text files. It can be complicated to perform certain data tasks, and difficult to remember all the commands necessary. There are some other tools that will greatly improve the ease and efficiency of working with data at the terminal.

The tools listed here will enable us to leverage SQL for data analysis.

These tools include:

Essential

  • csvkit - suite of tools for effective data manipulation at the terminal
  • sqlite-utils - tools for interacting with SQLite database

Highly recommended

  • cht.sh- collection of cheat sheets for bash and sql (and many others)
  • beekeeper studio - Open source SQL Editor and Database Manager that works with SQLite
  • rich-cli - fancy output in the terminal that includes 
###############################################
### Install all essential and recommended tools
###############################################

### install csvkit
sudo pip install csvkit


### install sqlite-utils
sudo pip install sqlite-utils


### install cht.sh
PATH_DIR="$HOME/bin"  # or another directory on your $PATH
mkdir -p "$PATH_DIR"
curl https://cht.sh/:cht.sh > "$PATH_DIR/cht.sh"
chmod +x "$PATH_DIR/cht.sh"


### install beekeeper Studio
# Install beekeeper GPG key
wget --quiet -O - https://deb.beekeeperstudio.io/beekeeper.key | sudo apt-key add -

# add our repo to your apt lists directory
echo "deb https://deb.beekeeperstudio.io stable main" | sudo tee /etc/apt/sources.list.d/beekeeper-studio-app.list

# Update apt and install
sudo apt update
sudo apt install beekeeper-studio


### install rich cli
pip install rich-cli

csvkit

csvkit is a suite of command-line tools for converting to and working with CSV, the king of tabular file formats.

csvkit installation instructions: https://csvkit.readthedocs.io/en/latest/tutorial/1_getting_started.html#installing-csvkit

sudo pip install csvkit

sqlite-utils

sqlite-utils is a library and command-line utility that helps create SQLite databases from an existing collection of data

sqlite-utils installation instructions


sudo pip install sqlite-utils

cht.sh

Unified access to the best community driven cheat sheets repositories of the world.

cht.sh installation instructions

# install cht.sh
PATH_DIR="$HOME/bin"  # or another directory on your $PATH
mkdir -p "$PATH_DIR"
curl https://cht.sh/:cht.sh > "$PATH_DIR/cht.sh"
chmod +x "$PATH_DIR/cht.sh"

Beekeeper Studio

A modern, easy to use, and good looking SQL client for MySQL, Postgres, SQLite, SQL Server, and more.

Beekeeper Studio installation instructions

# Install beekeeper GPG key
wget --quiet -O - https://deb.beekeeperstudio.io/beekeeper.key | sudo apt-key add -

# add our repo to your apt lists directory
echo "deb https://deb.beekeeperstudio.io stable main" | sudo tee /etc/apt/sources.list.d/beekeeper-studio-app.list

# Update apt and install
sudo apt update
sudo apt install beekeeper-studio

rich-cli

Rich-cli is a command line toolbox for fancy output in the terminal, built with Rich.

rich-cli installation instructions


pip install rich-cli