Migrate Docker Volumes, and the way to entry information inside a Docker Quantity? I will present you how one can simply again up and restore Docker Volumes and transfer the container information to a different host. #Docker #Volumes #HomeLab
Backup in Linux Servers – Docker Volumes, and Databases: https://youtu.be/JoA6Bezgk1c
Docker Documentation: https://docs.docker.com/storage/volumes/
My Docker CLI Cheat-Sheet: https://github.com/christianlempa/cheat-sheets
Observe me:
TWITTER: https://twitter.com/christianlempa
INSTAGRAM: https://instagram.com/christianlempa
DISCORD: https://discord.gg/christian-lempa-s-tech-community-702179729767268433
GITHUB: https://github.com/christianlempa
PATREON: https://www.patreon.com/christianlempa
MY EQUIPMENT: https://equipment.co/christianlempa
Timestamps:
00:00 – Introduction
00:43 – How Docker shops information in containers
01:36 – Docker Volumes or Binds
02:05 – Some instance Volumes and Binds
03:52 – Entry recordsdata inside named Volumes
04:21 – Backup, Restore and Migrate information
________________
All hyperlinks with “*” are affiliate hyperlinks.
supply
You haven’t compressed the tar, you need the z options.
Thanks for the video. The bit of info at 3:02 helped me find all the data for migration 🙂
Just use filezilla copy and paste, you linux guys love to complicate things
have you even checked the commands you wrote? shit does not work, busybox does not have bash, worked with sh and strip is not needed when restoring. and backing up and restoring involes not only this 2 commands, you need to create container, you need to stop it and bla bla bla.
What if there are multiple volumes attached to a container?
I've read several times that saving volumes can be problematic if it's performed at the same time as the container's data is being modified, rendering the backups unusable. Does it make sense to stop or pause the container beforehand? However, this would have the disadvantage of interrupting the container. What do you think about this?
Moin, cooles Video. ich nutze mit Absicht kein Portainer, weil das tatsächlich bei den Installationen in meiner VM unter Proxmox nur Probleme bereitet hat, zumindest als ich Paperless NGX installieren wollte. Habe es entsprechend mit Docker-Compose alleine durchgeführt. Jetzt habe ich das Problem, dass ich demnächst das Postgres upgraden müsste. Doch wie das genau funktionieren soll, ist mit zwei Wegen behaftet und als kein absoluter IT Profi möchte ich da keine Fehler machen. Ggf. hatte ich auch angedacht, wie ich das bestehende mit neuer DB auch direkt in Portainer packen würde bzw. mit Portainer den Docker erstellen würde. Doch muss ich zugeben, ist das mit docker immer so eine Sache, zumindest für mich.
Hey Chris sehr guter Content. Danke !
Hi, it doesn't work for the MySQL 8 container. … 2024-10-24T18:55:30.066298Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.40) starting as process 1
2024-10-24T18:55:30.077248Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2024-10-24T18:55:30.474671Z 1 [ERROR] [MY-012209] [InnoDB] Multiple files found for the same tablespace ID:
2024-10-24T18:55:30.474729Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 1 = ['lib/mysql/sys/sys_config.ibd', 'sys/sys_config.ibd']
2024-10-24T18:55:30.474752Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 4294967278 = ['lib/mysql/undo_002', 'undo_002']
2024-10-24T18:55:30.474764Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 4294967279 = ['lib/mysql/undo_001', 'undo_001']
2024-10-24T18:55:30.474776Z 1 [ERROR] [MY-012202] [InnoDB] Tablespace ID: 4294967294 = ['lib/mysql/mysql.ibd', 'mysql.ibd']
2024-10-24T18:55:30.474815Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Failed, retry may succeed.
2024-10-24T18:55:30.474858Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine
2024-10-24T18:55:30.475055Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2024-10-24T18:55:30.475095Z 0 [ERROR] [MY-010119] [Server] Aborting
Your Cheat-Sheets are awesome! Thanks for sharing!
Captain busy box doesn't elaborate at all what that should be if we don't use it super special busy box personal preference configuration he uses.
docker ps
docker exec -it 4334cf36a826 /bin/sh
ls -lh /var/opt/memos/
# then exit, back to host
exit
-COPY-contents of folder
docker cp 4334cf36a826:/var/opt/memos/memos_prod.db-shm ~/Desktop/memos_docker_save/
docker cp 4334cf36a826:/var/opt/memos/memos_prod.db-wal ~/Desktop/memos_docker_save/
docker cp 4334cf36a826:/var/opt/memos/memos_prod.db ~/Desktop/memos_docker_save/
-RESTORE-contents of folder
sudo docker cp ~/Desktop/memos_docker_save/memos_prod.db-wal 4334cf36a826:/var/opt/memos/memos_prod.db-shm
sudo docker cp ~/Desktop/memos_docker_save/memos_prod.db-wal 4334cf36a826:/var/opt/memos/memos_prod.db-wal
sudo docker cp ~/Desktop/memos_docker_save/memos_prod.db-wal 4334cf36a826:/var/opt/memos/memos_prod.db
You run an image not a container. You start a container. And –rm remove the container file system after using it. It doesnt remove other containers
Please do not use scp anymore. It is deprecated. Rsync is a more robust, faster and more reliable solution for copying files between servers.
Perfect Christian, exaclty what I was looking for! Will help me restore a clients data to their VMware VM containers. 🙂
or you could just compress the binded docker volume directory with tar and scp it to the binded location then run the docker stack with the binded volumes, i did that, after getting stuck. It worked beautifully, next create a cron script that backup the data to a backup server. Quick question if there are different addition of data into the container from different servers how do i combine the new changes into one server?
Hi there – I'm struggling to get my portainer instance to run via the web GUI- I have no idea how its become the way it has, i've not changed anything's actively and can only assume an update has been done passively. I've had a look around and I can't see any ports applied to the container – this maybe a red-herring but wanted to address this first just in case.
"docker ps -a | grep portainer
33f01ee62da7 portainer/portainer-ee:latest "/portainer" 3 months ago Restarting (2) 36 seconds ago portainer"
This is what im seeing… Please help if you can.
how i could make backup and restore when in 1 container i have many volumes. for example container with zabbix which consist from mysqlsslsnmptrapskeys volumes???
Clear and understandable. U r genius
no encuentro el Cheat-Sheets del que hablas
Github Link doesnt work, the page you showcase is nowhere to be found !!
Hi Christian, thanks so much for this video, super helpful. I am however not seeing the devops folder in order to test the backup and restore commands you use. Could you please advise?
Hi, wenn ich das auf meinem raspberry pi ähnlich mache wie du beschreibst (aber ohne busybox) und am Ende auf dem neuen Raspberry Pi "sudo docker load -i image.tar " eingebe, erscheint immer: no such file or directory
an was kann das liegen?
Ich starte meinen alten pi von SD karte. an USB hängt die neue M.2 SSD, auf welcher der neue pi schon funktionsfähig installiert wurde und welche einfach als usb-stick gemountet ist. dort kopiere ich die tar-datei hin.
Dann starte ich von dieser SSD und will die tar mit dem genannten befehl wieder verfügbar machen….. 🙁
I'm currently hosting a nextcloud container on a Raspberrypi and want to move it to a desktop computer that will function as my new server (both running Ubuntu). Is this the prefer method to move my data from one machine to another? Would you suggest just backing up the data and create a new container 'From the scratch'? I don't have any 'special' apps on Next cloud. I am using it exclusively as cloud storage
Good day Mr. Lempa, I hope you are doing well! I have a question for you. Suppose I had a Docker Swarm that, at the moment, consists of 4 raspberry pi 4s each with a 500gb m.2 SSD drive plugged into them. Suppose further that I am also running Portainer to manage my containers. Now suppose that what I want to achieve is data persistence across these 4 Nodes so that if, say, the node containing the data for my Gitea instance goes down and maybe even stays down, another node should start up the service and also still have the necessary data. Any idea on how I could achieve this?
You see what I mean?
How reliable is the backup if the container is running and accessing it while you make the tar? Hmmm….
thanx , youe ease my job
For Windows users without WSL2 replace $(pwd) with %cd% in the backup a container command:
docker run –rm –volumes-from CONTAINER -v %cd%:/backup busybox tar cvfz /backup/backup.tar CONTAINERPATH
Sir, half of you video got deleted in editing.
How can we run that migrate command for the container which is having more than one container path?
Dust against the mailer in in time setting
(assuming everything is binded) Alternatively, can I just copy the directory where I made folders for my containers in a tarball and export my container config as stacks, then in my fresh install – copy over your existing directories to where they once were and re-deploy the containers with stacks (.yaml files) that I exported?
you really should have taken the 3 minutes to also explain how to restore an image with the backed-up file. Becuase the command in your cheat sheat is really confusing, and its unclear if i have to create the image first, or what container names I should use in the command….
My Raspberry Pi Lite OS runs on a 64GB Micro SD card boot drive and I have a secondary 2TB SSD to store data in. I plan to install Jellyfin and PhotoPrism and NextCloud which requires installing Maria MySQL Data Base. What has me confused is how I specify all docker data storage for PhotoPrism, JellyFin and Nextcloud to take place on the 2TB SSD drive and not the 64GB Micro SD card boot drive? Do you have a video that shows correct docker Volume syntax that enables data to get stored onto a 2TB SSD secondary data drive?