User Tools

Site Tools


DBdumper, Network dump/restore system

DBdumper is a easy distributed network system for creating database dumps and restoring SQL databases, managed via Web interface

You can

You can manage databases easy like file in your file system.

  1. Visualize list of databases and dumps
  2. Create database dumps promptly and on cron-like schedule
  3. Quickly restore databases from backup copies
  4. Quickly transfer database(s) from one operation system/host to another
  5. Rename/Drop/Create empty database
  6. Easy add database to dump schedule

You can also

  1. Save dumps from any agent to any available file storage
  2. Restore a backup from a file storage to any dbms

Roles/user management, logging and other function can easy added.

The application code is object-oriented and a addition of functionality requires addition of methods without changing application architecture.

That is, the data storage/recovery model is “any to any”

Limitation into the moment

  1. One agent per one DBMS
  2. Just now only for PosgteSQL

Place

You can install dbagent to any system in network but I install agents into the same system where run database management system.

The dbstore can also install to any physical or virtual system with a reliable network band and conditions for storing backup copies

Format of dumps is gzip compressed SQL, dump name agreement is DBNAME–TIMESTAMP-TZ–SOURCE.HOST.sqlz

The kit of application consists of three components

  1. dbagent, with purpose dump/restore/rename database on database management system
  2. dbstorage, very simple dump file storage
  3. dbmaster, web front-end management console and DB dump scheduler (like UNIX cron but better =) )

All components interact with authenticated REST RPC over HTTPS.

Usage progress

  • 2018-Jan-12: Now I install first release DBdumper for regular backups 12 corporate finance databases what cover ~2000 staff.
  • 2018-Jan-24: Now I install next release for all corporate DBMS for validation stage
  • 2018-Jun-20: Current use-case is: 107 application databases in schedule, 17 database agents and 18 dump storage

RPC

I used a very simple kind of RPC over HTTP, isolated from the transport layer. Procedure parameters are sent as parameters GET/POST, responses is sent as JSON structure. In future it is very easy to replace to full JSON RPC 2.0

Sources

Bug

Please, send my to onborodin@gmail.com if found.

Component communication diagram

Short description

Master interact with Agent(s) using commands (Web REST RPC)

  1. list databases/users
  2. create/alter/drop the database/user
  3. dump the database and put dump to selected store
  4. restore database from selected store and dump to agent

Master with Store(s)

  1. list dump files
  2. delete files

Agent(s) with DBMS(es) interact using native SQL API

Screens

Typical use case

“I saw these people” =)

Literature

Install tips

apt-get -y install libjs-prettify
apt-get -y install libdbd-pg-perl
apt-get -y install libdbi-perl
apt-get -y install libapache-htpasswd-perl
apt-get -y install libfilesys-df-perl
apt-get -y install libev-perl
apt-get -y install libio-socket-ip-perl

wget http://wiki.unix7.org/_media/dist/jessi/libmojolicious-perl_7.43_dfsg-1_all.deb
wget https://wiki.unix7.org/_media/dist/dbagent_0.01.4-1_all.deb
wget https://wiki.unix7.org/_media/dist/dbstore_0.01.4-1_all.deb
dpkg -i *.deb && rm -v *.deb